In this article
Machine translation is generally most accurate for Spanish and other widely used languages and least accurate for languages with less training data, such as Haitian Creole, Armenian, Farsi and Swahili. That is the main pattern across 22 peer-reviewed studies published from 2011 to 2026, though not every study fits it: in a 2021 pharmacy study, Spanish drug counseling points fared worst of three languages. Most of the studies tested Google Translate or ChatGPT on hospital discharge instructions and other patient materials.
The newest AI tools come close to professional translators for written Spanish: in a 2025 study in JAMA Pediatrics, GPT-4o scored 98.3 out of 100 against 96.7 for human translators. But every study, including the most favorable, recommends human review for anything that matters, and speech is harder than text. In a 2026 study of Google Translate voice mode, 33.3% of translations of clinical speech had clinically significant errors, against 4.8% for qualified interpreters.
Key figures
33.3% vs 4.8%
translations of clinical speech with clinically significant errors: Google Translate voice mode vs. qualified interpreters (2026)
90% vs 10%
medical phrases Google Translate got right: Portuguese vs. Swahili (2014)
98.3 vs 96.7
quality scores for written Spanish: GPT-4o vs. professional translators (2025)
22
peer-reviewed studies summarized, published 2011 to 2026
The studies at a glance
| Source | Tool | Languages | Accuracy | Potentially harmful errors |
|---|---|---|---|---|
| Khanna et al., 2011, J Hosp Med | GT | Spanish | Any error in 39% of GT sentences vs 22% of professional | Severe errors 4% vs 2% (not significant) |
| Patil and Davies, 2014, BMJ | GT | 26 languages | 57.7% of translations correct; Swahili 10%, Portuguese 90% | Not measured as a rate |
| Turner et al., 2015, JMIR Public Health Surveill | GT plus human editing | Chinese | Raters consistently preferred the human translation | Not reported |
| Chen et al., 2016, JMIR Diabetes | GT | Spanish, Chinese | Spanish close to human; Chinese worse (6 sentences) | Some Chinese sentences risked delayed care |
| Khoong et al., 2019, JAMA Intern Med | GT | Spanish, Chinese | 92% of Spanish and 81% of Chinese sentences accurate | 2% of Spanish and 8% of Chinese sentences |
| Rodriguez JA et al., 2021, J Gen Intern Med | GT | Portuguese to English | Non-inferior to human on 24 of 25 comprehension questions | Not reported |
| Taira et al., 2021, J Gen Intern Med | GT | 7 languages | 82.5% kept meaning; Spanish 94%, Armenian 55% | Not reported as a rate |
| Cornelison et al., 2021, Am J Health Syst Pharm | GT | Arabic, Chinese, Spanish | Drug counseling points: Chinese 76.5%, Arabic 54.1%, Spanish 38.2% | 29.1% of inaccurate translations highly significant or potentially life-threatening |
| Hudelson and Chappuis, 2024, J Gen Intern Med | Voice apps | 18 languages | Clinicians met goals in 82.7% of visits, satisfied in 53.8% | Not measured |
| Brewster et al., 2024, Pediatrics | GT, ChatGPT-4 | Spanish, Portuguese, Haitian Creole | Similar to professional for Spanish and Portuguese; worse for Haitian Creole | Haitian Creole: ChatGPT 33.3%, GT 23.3%, professional 8.3% |
| Briva-Iglesias et al., 2024, MonTI | GT, GPT-4, Vicuna | Spanish, Catalan, Turkish, Portuguese | Legal contract: GPT-4 comparable to or slightly better than GT, except Turkish (GT better) | Not assessed |
| Ray et al., 2025, JAMA Pediatr | GPT-4o | Spanish | Quality score 98.3 vs 96.7 for human translators (equivalent) | Critical errors 6 vs 8 for humans (not significant) |
| Ugas et al., 2025, Patient Educ Couns | GT | Spanish, Portuguese, Punjabi, Chinese, Vietnamese | Human better in every language | Severity rated; GT scored high |
| Altakhaineh et al., 2025, Int J Lang Law | ChatGPT | Into Arabic | Legal contracts: human better on every criterion (one expert's review) | Not assessed |
| Carreras Tartak et al., 2025, J Emerg Med | GT | 26 languages | 95% overall; under 90% in Amharic, French, Oromo, Portuguese, Yoruba | Not reported |
| Kong et al., 2026, BMJ Qual Saf | GT, ChatGPT-4 | Spanish, Chinese, Russian | Sentences: Spanish 96 to 97%, Chinese 90 to 95%, Russian 80 to 89% | 1% or less of sentences |
| Carreras Tartak et al., 2026, JMIR Form Res | Claude Sonnet 3.5 | Spanish | 4.9 of 5 overall from physicians and interpreters | None unacceptable after review (1 of 100 flagged by one reviewer) |
| Wang et al., 2026, CJEM | GT, ChatGPT-4 | Arabic, Chinese | ChatGPT more accurate in both languages | No difference in safety |
| Wu et al., 2026, West J Emerg Med | GT, Copilot, ChatGPT-4 | Albanian, Portuguese, Vietnamese | No significant difference overall | 5 potentially harmful errors from GT and Copilot combined, 0 from ChatGPT-4 |
| Rodriguez G et al., 2026, Acad Emerg Med | GT, ChatGPT-4o | Spanish, Portuguese, Chinese | Not inferior to professional interpreters on most measures | No difference in clinically significant errors |
| Chao et al., 2026, J Med Internet Res | ChatGPT, GT, UD Talk | Chinese to Japanese | Median 5 (top of a 0 to 5 scale) for ChatGPT vs 2 for GT | Not reported |
| Feinberg et al., 2026, Healthcare | GT voice mode | Amharic, Arabic, Burmese, Spanish, Swahili, Vietnamese | More errors than qualified interpreters | 33.3% of translations vs 4.8% for interpreters |
| WHO interpretation team, 2025 (staff report) | Wordly AI interpreting | 6 UN languages | 1 of 90 interpretations passed; average score 46% | Every interpretation had at least one reputational risk |
What most of the studies show
- Accuracy depends on the language. Spanish and Western European languages score highest; languages with less online text score lowest. Systematic reviews in 2018 (Dew et al.) and 2025 (Karakus et al.) reached similar conclusions.
- Simple sentences translate better than technical ones. Medication instructions and counseling points produced some of the worst results.
- Tools have improved. The same research group found Google Translate's sentence accuracy rose from 92% to 96% in Spanish and from 81% to 90% in Chinese between its 2019 and 2026 studies.
- Small error rates add up. In the 2026 study, 95% of Chinese sentences translated by ChatGPT were accurate, yet 24% of complete Chinese instruction sets contained at least one error; for Google Translate in Russian, 66% did.
- No study tested whether patients were actually harmed. They measure translation quality, usually on small samples, and say so.
Spanish versus other languages
The gap between languages is large and appears in most multi-language studies. In the 2014 BMJ study, Google Translate got 90% of medical phrases right in Portuguese and 10% in Swahili, where "Your child is fitting" came back as "Your child is dead." In Taira et al. (2021), emergency department instructions kept their meaning 94% of the time in Spanish, 90% in Tagalog, 82.5% in Korean, 81.7% in Chinese, 77.5% in Vietnamese, 67.5% in Farsi and 55% in Armenian, where an instruction to take ibuprofen became an instruction to take an anti-tank missile.
Newer tools narrow the gap but do not close it. In a 2024 study of 20 standardized pediatric discharge instructions, potentially clinically significant errors in Haitian Creole were more common with ChatGPT (33.3%) and Google Translate (23.3%) than with professional translations (8.3%). In the 2026 voice study, Google Translate's error rate was 42.9% in less commonly spoken languages and 14.3% in common ones.
Newer AI tools: ChatGPT, GPT-4o and Claude
Studies of large language models since 2024 report results close to professional translation for Spanish. Ray et al. (2025) found GPT-4o equivalent to human translators on 20 real pediatric instruction files, with fewer major errors. Rodriguez G et al. (2026) found Google Translate and ChatGPT-4o not inferior to professional medical interpreters on most measures in Spanish, Portuguese and Chinese, and evaluators often mistook the machine output for a professional's. A 2026 study of Claude Sonnet 3.5 scored Spanish discharge instructions 4.9 out of 5.
All of these authors still call for human review. The 2025 GPT-4o study says human review "remains essential in health care" and frames the tool as a way to free up translation resources for other languages. The 2026 BMJ Quality and Safety study recommends limiting machine translation without professional review to low-stakes written communication from English to Spanish.
Machine interpreting: speech is harder
Few studies test spoken language, and those that do show bigger gaps than written translation. In Feinberg et al. (2026), recorded physician sentences, read into Google Translate voice mode by a study investigator, produced clinically significant errors in 28 of 84 translations (33.3%), against 4 of 84 (4.8%) for qualified in-person interpreters. In a 2025 report by the World Health Organization interpretation team, posted online by the American Translators Association, only 1 of 90 AI interpretations of World Health Assembly speeches reached the passing score; in one, hepatitis became Ebola.
In real consultations in Geneva, clinicians using voice translation apps achieved their goals in 82.7% of visits but were satisfied with the communication in only 53.8% (Hudelson and Chappuis, 2024). The authors concluded that professional interpreters remain the gold standard.
Machine translation in legal settings
Research on legal text is thinner. A 2024 study of a legal contract found human raters judged GPT-4 comparable to or slightly better than Google Translate in Spanish, Catalan and Portuguese, while a 2025 study of contracts translated into Arabic found human translators better on every criterion, including legal terminology, in one expert's qualitative review.
Courts have already ruled on it. In United States v. Cruz-Zamora (D. Kan. 2018), a trooper used Google Translate to ask a driver for consent to search the car. "Can I search the car?" came out as "¿Puedo buscar el auto?", which Google Translate renders back into English as "Can I find the car." The court held that it is not reasonable for an officer to rely on Google Translate to obtain consent to a warrantless search, especially when more reliable options exist, and suppressed the evidence. In California, 35 legal aid and community organizations asked a Judicial Council language access subcommittee in August 2026 to suspend automated translation in state courts, citing examples such as "DUI" translated as "IUD," according to the Legal Aid Foundation of Los Angeles.
For depositions, the same caution applies: a machine cannot take the interpreter's oath. See consecutive vs. simultaneous interpreting at depositions and sight translation of exhibits.
What the rules say
- HHS, under Section 1557 of the Affordable Care Act, requires a qualified human translator to review machine translation when the text is critical to a person's rights, benefits or meaningful access, when accuracy is essential, or when the material is complex, non-literal or technical (45 CFR 92.201(c)(3)). See Section 1557 language access.
- The National Center for State Courts (2025) advises that courts should never use machine translation for courtroom interactions or to convey legal or procedural information, and limits it to brief, non-complex, out-of-court uses such as wayfinding, or to drafts that a professional legal translator reviews.
- An NCSC and Thomson Reuters Institute consortium (June 2025) answered the question "Can AI replace human court interpreters?" with "No" for real-time interpretation in court proceedings.
- The U.S. Attorney General's July 14, 2025 memo implementing Executive Order 14224 encourages federal agencies to consider "responsible use" of AI and machine translation, without defining it. It does not change the HHS rule above. See the federal language access tracker.
- The American Translators Association (2018) says that if reliable and secure translation is desired, machine translation should not be used without the ongoing involvement of professional translators, and warns that free public tools capture the text they process for training.
- The Interpreting SAFE-AI Task Force (2024) says any use of AI interpreting requires informed consent, a way to switch to a human interpreter at any time, and a process to report errors.
When machine translation is reasonable, and when it is not
Taken together, the studies and rules point to a simple line. Machine translation can help with low-stakes, simple text in a high-resource language, such as directions to a building, or as a first draft that a qualified translator then reviews. It should not be used on its own for consent forms, medication instructions, discharge instructions, legal documents, court or deposition testimony, or anything in a language where it is known to perform poorly.
Free online tools also raise a confidentiality problem for medical and legal text, since the text is sent to a third party. For documents that will be filed, relied on or signed, use a certified translation by a qualified translator.
About this review
- Studies were found through PubMed and journal databases and checked against the published record; figures are copied from each paper's abstract or full text. Where a paper's text and tables differ, the table is used.
- Most studies are small, from a single pamphlet or contract to 100 source documents, and none is a randomized trial of patient outcomes. Tools change quickly, so each result describes the version tested at the time.
- The 2011 to 2016 studies predate neural Google Translate, introduced in late 2016. They are included for history and should not be read as describing today's tools.
- The WHO report is a staff team's evaluation of one vendor in a conference setting; the American Translators Association, not WHO, posted it online.