Editor’s note: Dr. Arle Lommel is a speaker for ODSC East 2022. Be sure to check out his talk, “How Can We Make Machine Translation Responsive and Responsible?” to learn more about responsive MT!
Machine translation (MT) has become ubiquitous as a technology that enables individuals to access content on-demand in real-time that is written in languages they do not speak. However, contrary to recent press releases that have said it has surpassed human quality, the results in practice suggest that it has a long way to go. One of the biggest challenges current-generation neural MT (NMT) faces is that its engines are not easily adaptable and are fundamentally unable to respond to context or extra-linguistic knowledge that human translators routinely deal with. In addition, NMT’s improvements have largely been in terms of fluency (how natural the output sounds) rather than accuracy (how well the translated text represents the content of the source text). This discrepancy in improvement often increases the risk that critical errors may remain undetected simply because they are readable and sound plausible.
The next step forward is to build “responsive MT”: systems that can take advantage of embedded metadata about a wide variety of topics and use them to preferentially use the most relevant training data. This metadata includes factors such as text formality, client, product line, text type (e.g., marketing, legal, FAQ, subtitle), intended audience, attributes about the speaker or author, date of authorship, human quality judgments, etc.
Next Evolutionary Step in MT: Responsive MT
Responsive MT refers to systems that are better able to respond to context and stakeholder requirements so that they deliver situationally relevant results. Not only does responsive MT build on trends from the seven-decade history of MT technology, but it also increases flexibility and suitability for a wide diversity of use cases. The magic behind the scenes involves increasingly sophisticated metadata and context that allow the emergence of “polymorphic engines,” which adapt dynamically to the content they translate in a way that current-generation MT systems cannot.
Since the early 2000s machine translation developers have customized their engines by using training data specific to an organization, subject field (also known as a “domain”), or type of document. However, this approach is relatively inflexible when faced with content that crosses domains, such as the inclusion of a legal notice in a marketing document. It also does not help problems that arise because MT looks only at single sentences in isolation. To address these limitations, responsive MT approaches focus on broader units of text (such as paragraphs), adjust in response to feedback from users, make use of metadata about texts, and factor in stakeholder requirements. Taking this approach will enable MT to be more contextually relevant and deliver results more likely to be correct.
CSA Research has identified four problems and solutions that responsive MT will address, sorted from those easiest to solve to those that require more development.
Problem #1: MT struggles with terminology and other inputs. Although developers may claim that their engines can incorporate terminology, our surveys demonstrate that this capability often fails to meet expectations. Our observations show that some MT systems still struggle with simple glossary intake and usage. Similarly, today’s engines cannot integrate requirements from style guides, and many cannot incorporate additional translated data without a full retraining cycle.
Solution: Adaptive machine translation, which uses engines that update their training data in real-time based on linguists’ corrections, helps resolve some of these issues. However, developers need to make it easier and more intuitive for implementers to improve results in real-time.
Problem #2: MT’s focus on a domain does not reflect real-world content. At present, MT software can access multiple trained solutions but typically deploy only one engine per document. This approach increases consistency but does not account for cases such as a marketing document that contains a legal disclaimer, or a service manual that embeds product information. As a result, each document receives only a slice of the industry or organizational knowledge available to its narrowly trained MT engine.
Solution: Ideally, each segment would be translated with the most relevant training data based on the kind of content it is, not on the domain limitations of a specific engine. That means that MT for a product description on a website might require access to engines trained for marketing, product information, and legal compliance.
Problem #3: MT misses contextual clues. Because MT focuses on individual segments in isolation, it struggles with elements such as determining which pronouns to use in languages with gender distinctions or correctly determining which meaning of a homonym to apply – for example, knowing whether the German word Gericht means “court” or “dish” in a given context. Similarly, the ability to determine the proper subject of a sentence when translating from a language where subjects may not be stated will result in far more useful translations.
Solution: Developers should expand the focus of MT beyond segments to what comes before or after. Some are already experimenting with whole-document engines, but even evaluating one segment before and after the current one can deliver significant advantages in improving grammar and cohesion in translated texts. Other developers are building recursive, multi-pass architectures that can go back and rewrite output based on clarification and additional knowledge within the content they are translating, much as human translators will revise previously rendered content.
Problem #4: MT quality is plateauing. By 2019 we saw signs that the initial rapid improvements in neural MT (NMT) have made compared to rule-based and statistical systems have started to slow down, although developers continue to obtain incrementally better results. Further changes require increasingly sophisticated technologies. In addition, poor training and data hygiene practices mean that many efforts fall short of expectations and few organizations currently collect the kinds of metadata that would help them improve training. Until now, developers have tended to rely on access to greater quantities of training data to drive quality forward, but it has become increasingly difficult to find “pure” sources that have not themselves been produced by NMT engines.
Solution: Incorporating sophisticated metadata into the translation process will enable MT to mimic the decision-making capabilities of humans more closely and address the problems listed here. The needed metadata will cover everything from product lines to speech formality, dialect, reading level, subject field, and text type to intent, sentiment, and other communicative features. The challenge will be to keep a “certified pure” stream of metadata that can be used for training tools and processes while avoiding situations in which automatically generated metadata of dubious quality is used indiscriminately.
Providers that can deliver effective solutions that enable MT to adapt responsively to contexts and stakeholder requirements will have a first-mover advantage that will set them apart and ensure that language companies and professional translators alike will be able to rely on their solutions and adapt them for an ever-growing array of requirements. At the same time, competitors will need to work together to define common formats and approaches to the exchange and use of data so that they all benefit together from developments.
Find Out More
In his presentation at ODSC East, Dr. Arle Lommel, senior analyst at CSA Research, will outline the types of metadata that need to be encapsulated and the best practices for gathering them in preparation for the release of responsive MT systems. It will also discuss how these changes are likely to affect technology and translation providers and the new career opportunities that will appear for language professionals.
About the Author/ODSC East 2022 Speaker on Responsive MT:
Dr. Arle Lommel is a senior analyst with independent market research firm CSA Research. He is a recognized expert in translation quality processes and interoperability standards. Arle’s research focuses on translation technology and the intersection of language and artificial intelligence as well as the value of language in the economy. Born in Alaska, he holds a PhD from Indiana University. Prior to joining CSA Research, he worked at the German Research Center for Artificial Intelligence (DFKI) in its Berlin-based language technology lab. In addition to English, he speaks fluent Hungarian and passable German, along with bits and pieces of other languages.
