Case Objectives

• Define limited English proficiency.
• Understand the principal approaches to machine translation.
• Review the way machine translation systems are evaluated for quality.
• Discuss the current role of machine translation systems in health care settings.
• Describe the limitations of machine translation system use in health care.

The Case

A 54-year-old Spanish-speaking woman with depression and anxiety disorder presented to an urgent care clinic complaining of headache and worsening dizziness. She had seen her primary care physician a few weeks earlier for the same symptoms and, after a normal physical examination, she was prescribed meclizine for symptomatic treatment of the dizziness. She now presented with a worsening headache and dizziness. During the interview with the urgent care provider, she was anxious about the nature and degree of her symptoms. The history was obtained using a telephone Spanish medical interpreter from a medical interpreter/translator company contracted by the hospital.

The patient had a normal physical examination, including a normal neurologic exam, during the urgent care visit. However, given the worsening symptoms and persistent headache, the provider ordered an urgent magnetic resonance image (MRI) of the brain. The MRI was completed later that day and officially interpreted as normal, with no concerning findings and no explanation for her symptoms. Given the patient's anxiety, the urgent care provider wanted to inform her of the results that day. He called the phone number in the record but there was no answer and no opportunity to leave a voicemail. He decided to send a letter to ensure receipt of the results.

However, he was worried that the patient would not be able to read the letter or have anyone to translate it and decided to use Google Translate to do so himself. He typed the following into Google's translator: "The results of your recent MRI were normal, making an infection, mass, or stroke unlikely causes of your symptoms. Please make sure to follow up with your primary care provider and call the clinic with any questions." Unfortunately, the Google-translated output was such that, when the patient received the letter she could not understand the statement. She interpreted the wording to mean that the MRI showed a mass, infection, or stroke. She became very anxious and immediately returned to urgent care, bringing the letter with her.

Another urgent care provider (also English-speaking) saw her and had difficulty obtaining a clear history from the anxious and tearful patient. Even though the provider read the normal findings of the MRI included in the patient's electronic chart and described this to the patient using a phone interpreter, she remained distressed and upset, complaining of a severe headache and worsening dizziness. He referred the patient to the emergency department (ED) for further evaluation. In the ED, neurology was consulted and concluded that the patient's symptoms were likely benign positional vertigo (a common condition that leads to intermittent dizziness) with amplification of her symptoms by her anxiety. She was treated with an Epley maneuver, taught to perform this maneuver at home, and discharged.

The second urgent care provider was troubled by the case. He took the letter to an in-person medical interpreter who read the letter and noted that the language and syntax used by Google Translate was confusing and could indeed have been understood to relay serious findings. The miscommunication led to patient distress, frustration, and unnecessary visits to the urgent care clinic and ED. Both urgent care providers were left wondering whether software programs that translate from one language to another are accurate enough to be used in health care.

The Commentary

by Anne M. Turner, MD, MLIS, MPH

This case illustrates how poor translation can lead to miscommunication, which can result in adverse events including patient distress, frustration, and possibly unnecessary health care visits or resource utilization. Although the first urgent care provider was attempting to meet an important responsibility—adequately communicating with a patient with limited English proficiency (LEP)—confusion ensued. Translation tools, such as Google Translate, are not adequate in their current form for translating clinical information and should not have been used without review from a professional translator. The quality of current machine translation, even for common languages such as Spanish, is not high enough to use safely in clinical settings without careful human editing and oversight.

The Need for Machine Translation

The Spanish-speaking patient in this case is not alone. In 2013, it was estimated that more than 350 languages were spoken in the United States, and approximately 60.3 million people residing in the US speak a language other than English at home.(1) Among these individuals, about 41% (25.1 million) are considered to have LEP, meaning they do not speak English as their primary language and have a limited ability to read, speak, write, or understand English. Overall, 8% of the total US population aged 5 and older have LEP.(1)

In health care settings, providers are encouraged to optimize communications in situations in which there is language incongruence by using trained professionals to interpret verbal communications and translate written communications. Federal policies require that health care organizations that receive federal funding take reasonable steps to provide meaningful language access to LEP populations.(2-6) Depending on the clinical situation, meaningful language access means providing oral language assistance services or written translation. However, the lack of trained medical interpreters and translators, as well as perceived expense and time required to engage professional translators, often prevents providers from using medical translators. Machine translation tools, which are increasingly used in business and travel, appear to provide a low-cost and real-time alternative. Nevertheless, these tools alone are not yet sufficient for use in clinical settings without bilingual human oversight.

What Is "Machine Translation"?

Machine translation uses computer algorithms to automate translation from one language to another. There are two principal approaches used for machine translation: natural language processing, also called rule-based translation, and statistical machine translation. Applying linguistic knowledge of syntax, morphology, and semantics, rule-based translation creates computational rules for translating from a source language to another language. The other approach, statistical machine translation, uses large corpora of previously translated documents to train the system to translate words or phrases based on statistical probability. Google Translate and Microsoft Translator are freely available online translation tools that were developed based on the statistical machine translation approach.

In general, statistical machine translation has performed better than rule-based translation at language translation when large corpora of translated documents related to the domain, or field, in question are available to develop the statistical methodology. However, statistical machine translation–based translation is less accurate for low-resource languages (i.e., languages for which there are insufficient parallel translated text available for training) and for domain areas, such as medicine, where a sufficient number of translated documents are not available. Rule-based tools specifically designed for a given language pair and domain generally perform better with less common languages or with highly technical areas. In practice, most commercial translation systems use hybrid approaches.

Accuracy of Machine Translation

Machine translation has not been tested extensively in the clinical domain, and research on the accuracy of machine translation for medical records and health promotion materials suggests mixed results.(7-10) As with domains outside medicine, the accuracy of machine translation increases when a large corpus of high-quality translations in the domain area is available to learn from, and when the two languages are closely related linguistically (e.g., Spanish and Italian). Our research and the research of others has shown translation of health promotion materials from English to Spanish to be fairly accurate when translated by freely available online software, such as Google Translate or Microsoft Translator.(8,11) Nevertheless, these translations still have many errors and require human editing to ensure accuracy. On the other hand, translation of health promotion materials from English to Chinese using the same machine translation tools is highly inaccurate and, even with post-editing by professional translators, machine translation was found to be inferior to human translations.(10)

Machine Translation in Health Care

As this case illustrates, serious errors and misunderstandings can result if machine translation tools are used without the careful review of a bilingual translator familiar with the domain. The health domain has its own vocabulary, requires clinical communications to be accurate and precise, and lacks high quality translations to train the system. Although machine translation systems are constantly improving, there are currently no examples of highly effective and accurate application of machine translation in clinical settings.

The few published studies that have assessed machine translation in health care settings have involved small pilot studies describing and evaluating specific translation systems or source materials.(12-14) No controlled trials of machine translation use in clinical settings have been reported, and no studies have correlated translation with clinical outcomes. While machine translation tools show promise, they currently do not show sufficient accuracy and fluency to warrant widespread deployment in a health care setting.

Speech-to-speech translation tools that combine a speech recognizer with machine translation are also being tested for use in clinical settings.(13-15) These systems rely on a multistep process that typically includes speech-to-text recognition and machine translation. As with machine translation of text, the availability of more advanced statistical models, larger data resources, and more powerful hardware is contributing to their rapid development. Although these preliminary investigations have shown promise, there has been no widespread deployment of these tools in a health care context.

Provider Responsibility

Several national policies encourage or require the use of language services in public agencies serving LEP populations. Statutory requirements under Title VI of the Civil Rights Act of 1964 prohibit discrimination based on national origin (2), which has been interpreted in many cases to include one's preferred language. This means that agencies that receive federal funding must provide meaningful language assistance to LEP populations.(3) The Office of Minority Health, a federal agency that advises the Department of Health and Human Services (HHS) on key health issues affecting racial and ethnic minorities, has developed the Culturally and Linguistically Appropriate Services (CLAS) framework to improve language assistance for individuals with LEP. The CLAS framework includes four standards referencing availability of verbal and written information in one's preferred language.(4)

The Affordable Care Act (ACA) of 2010 made several requirements of health plans and providers, including that patient communication is in a language that the intended audience can readily understand and use, and that qualified health plans provide notices of appeal and summaries of benefits and coverage in a culturally and linguistically appropriate manner.(5) The HHS regulatory standards pursuant to the ACA require health insurance exchanges to provide for oral interpretation, written translation, and taglines in non-English languages informing patients about such services, all at no cost.(6) Additionally, states have their own laws and regulations about the provision of language services. Since at this time machine translation is not accurate enough to be applied in the health care setting, providers and institutions should meet these requirements by offering in-person interpreters where appropriate and using professional translator services to convert written materials.

An excellent resource for improving communications for LEP individuals in hospital settings is Improving Patient Safety Systems for Patients With Limited English Proficiency: A Guide for Hospitals.(16)

The Future of Machine Translation

A new class of statistical machine translation models based on neural networks has recently emerged. A neural network is a computer architecture that has interconnected processing elements similar to the human brain in terms of adapting to rules and experiences.(17) Translation systems that use the neural network approach are reported to be more efficient and accurate than classic statistical machine translation systems.(18) The application of this approach to the highly domain-specific materials encountered in health care settings has not yet been evaluated.

In summary, although machine translation is rapidly evolving and shows great promise for use in clinical settings, as the case above illustrates, poor translations can lead to miscommunication, which can result in adverse consequences. Miscommunications from poor translation can lead to delayed diagnoses and treatment, causing injury and even death.(19,20) Providers should continue to utilize and promote medical translation services to ensure accurate communication with LEP individuals in clinical settings.

Today, the most promising uses of machine translation include usages where it is important to quickly translate large amounts of materials for a first-pass translation that is then corrected by a proficient human translator. Still we can look forward to a future state when machine translation is seamlessly integrated into provider workflow to enhance communication with LEP patients.

Take-Home Points

• A growing number of languages are spoken in the United States, and there is a tremendous need for providing access to multilingual health information for ensuring accurate communication in health care settings.
• Federal and state regulations require that language-appropriate communication be provided in health care. It is the responsibility of the health care system to ensure that adequate communication occurs between health care providers and patients with limited English proficiency.
• Current health care translation services rely primarily on human translators, which are expensive and may not be available for low-resource languages.
• To meet the high demand for language translation in health settings, several machine translation systems have been evaluated for use in translating health materials or for improving communications during clinical visits.
• Although they have great potential for improving the efficiency of translation for health, current machine translation technologies, while constantly improving, are not safe to use in clinical settings without experienced bilingual human oversight.

Anne M. Turner, MD, MLIS, MPH Associate Professor, Department of Health Services, School of Public Health Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, Seattle, WA

Faculty Disclosure: Dr. Turner has declared that neither they, nor any immediate members of their family, have a financial arrangement or other relationship with the manufacturers of any commercial products discussed in this continuing medical education activity. In addition, the commentary does not include information regarding investigational or off-label use of pharmaceutical products or medical devices.

References

1. US Census Bureau. Detailed languages spoken at home and ability to speak English for the population 5 years and over: 2009–2013. [Available at]

2. Civil Rights Act of 1964, Pub.L. 88-352, 78 Stat. 241 (1964).

3. Guidance to Federal Financial Assistance Recipients Regarding Title VI Prohibition Against National Origin Discrimination Affecting Limited English Proficient Persons. Washington, MD: US Department of Health and Human Services, Office for Civil Rights; 2002. [Available at]

4. National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care. Rockville, MD: US Department of Health and Human Services, Office of Minority Health. [Available at]

5. Patient Protection and Affordable Care Act, 42 U.S.C. § 18001 et seq. (2010).

6. Teitelbaum J, Cartwright-Smith L, Rosenbaum S. Translating rights into access: language access and the Affordable Care Act. Am J Law Med. 2012;38:348-373. [go to PubMed]

7. Zeng-Treitler Q, Kim H, Rosemblat G, Keselman A. Can multilingual machine translation help make medical record content more comprehensible to patients? Stud Health Technol Inform. 2009;160:73-77. [go to PubMed]

8. Kirchhoff K, Turner AM, Axelrod A, Saavedra F. Application of statistical machine translation of public health information: a feasibility study. J Am Med Inform Assoc. 2011;18:473-478. [go to PubMed]

9. Patil S, Davies P. Use of Google Translate in medical communication: evaluation of accuracy. BMJ. 2014;349:g7392. [go to PubMed]

10. Turner AM, Dew KN, Desai L, Martin N, Kirchhoff K. Machine translation of public health materials from English to Chinese: a feasibility study. JMIR Public Health Surveill. 2015;1:e17. [go to PubMed]

11. Khanna RR, Karliner LS, Eck M, Vittinghoff E, Koenig CJ, Fang MC. Performance of an online translation tool when applied to patient educational material. J Hosp Med. 2011;6:519-525. [go to PubMed]

12. Ehsani F, Kimzey J, Zuber E, Master D. Speech to speech translation for nurse patient interaction. Paper presented at: 22nd International Conference on Computational Linguistics; August 23, 2008; Manchester, UK; 54-59. [Available at]

13. Soller RW, Chan P, Higa A. Performance of a new speech translation device in translating verbal recommendations of medication action plans for patients with diabetes. J Diabetes Sci Technol. 2012;6:927-937. [go to PubMed]

14. Seligman M, Dillinger M. Evaluation and revision of a speech translation system for healthcare. Proceedings of the 12th International Workshop on Spoken Language Translation. Da Nang, Vietnam; December 2015. [Available at]

15. Chen X, Acosta S, Barry AE. Evaluating the accuracy of Google Translate for diabetes education material. JMIR Diabetes. 2016;1:e3. [Available at]

16. Betancourt JR, Renfrew MR, Green AR, et al. Improving Patient Safety Systems for Patients With Limited English Proficiency: A Guide for Hospitals. Agency for Healthcare Research and Quality; September 2012. AHRQ Publication No. 12-0041. [Available at]

17. Stergiou C, Siganos D. Neural Networks. 

18. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. Proceedings of the International Conference on Learning Representations. San Diego, CA; May 2015. [Available at]

19. Goenka PK. Lost in translation: impact of language barriers on children's healthcare. Curr Opin Pediatr. 2016;28:659-666. [go to PubMed]

20. Flores G. Lost in translation? Pediatric preventive care and language barriers. J Pediatr. 2006;148:154-157. [go to PubMed]