Research by four Cyndx data scientists will be published in Findings of ACL: EMNLP 2020, an online publication from the Association of Computational Linguistics.
The paper, “Query-Key Normalization for Transformers,” identifies an improved mechanism for training artificial neural networks to translate low-resource languages. Written by Alex Henry et al., it will be available on the ACL anthology by mid-November, 2020.
A Prestigious Accomplishment for Cyndx Data Scientists
Intended as an online companion to the 2020 Conference on Empirical Methods in Natural Language Processing, Findings will feature peer-reviewed research of particular quality, novelty, rigor, and experimental soundness. It will cover a broad range of topics related to natural language processing (NLP), such as computational social science and social media; dialogue and interactive systems; discourse and pragmatics, etc.
The paper by the Cyndx team is one of only 520 chosen for publication, from 3677 submissions.
“Our paper shares a key finding from the team’s research into the Transformer, a deep learning architecture that has become the basic building block for NLP deep learning models across a wide variety of applications,” explains Alex Henry, Director of AI Research. “The team’s research lets the Transformer learn more nuanced patterns of “attention" — numerical weights between words in a piece of text, capturing their relative importance to one another.”
Advancing State-of-the-Art in LRLT
In their paper, the team focuses on low-resource language translation, which affords a straightforward demonstration of the technique without compromising current R&D of Cyndx products.
“We were able to establish state-of-the-art on several benchmarks,” says Henry. “The technique can be used to improve the performance of translation systems for languages with limited training data, like Oromo and Sinhala.”
Such an application may also have significant social value: better machine translation can benefit a variety of settings, including language preservation, education, political process monitoring, and emergency response.
“We’re excited about the work we’re doing at Cyndx,” Henry says. “A lot of our research focuses on language models and transformers — it’s fun to share the work we’re doing with the rest of the world.”
Accelerate M&A deal-sourcing
Simplify your capital raise with deep learning.
Securely manage your cap tables for $10 per month.