Performance of an online translation tool when applied to patient educational material.
BACKGROUND
Language barriers may prevent clinicians from tailoring patient educational material to the needs of individuals with limited English proficiency. Online translation tools could fill this gap, but their accuracy is unknown. We evaluated the accuracy of an online translation tool for patient educational material.


METHODS
We selected 45 sentences from a pamphlet available in both English and Spanish, and translated it into Spanish using GoogleTranslate™ (GT). Three bilingual Spanish speakers then performed a blinded evaluation on these 45 sentences, comparing GT-translated sentences to those translated professionally, along four domains: fluency (grammatical correctness), adequacy (information preservation), meaning (connotation maintenance), and severity (perceived dangerousness of an error if present). In addition, evaluators indicated whether they had a preference for either the GT-translated or professionally translated sentences.


RESULTS
The GT-translated sentences had significantly lower fluency scores compared to the professional translation (3.4 vs. 4.7, P < 0.001), but similar adequacy (4.2 vs. 4.5, P = 0.19) and meaning (4.5 vs. 4.8, P = 0.29) scores. The GT-translated sentences were more likely to have any error (39% vs. 22%, P = 0.05), but not statistically more likely to have a severe error (4% vs. 2%, P = 0.61). Evaluators preferred the professional translation for complex sentences, but not for simple ones.


CONCLUSIONS
When applied to patient educational material, GT performed comparably to professional human translation in terms of preserving information and meaning, though it was slightly worse in preserving grammar. In situations where professional human translations are unavailable or impractical, online translation may someday fill an important niche.
Division of General Internal Medicine, University of California, San Francisco, San Francisco, California 94143-1211, USA. raman.khanna@ucsf.edu
Raman R
RR
Khanna
Leah S
LS
Karliner
Matthias
M
Eck
Eric
E
Vittinghoff
Christopher J
CJ
Koenig
Margaret C
MC
Fang
D003142:Communication; D004521:Educational Measurement; D004522:Educational Status; D007722:Health Knowledge, Attitudes, Practice; D057220:Health Literacy; D006801:Humans; D007802:Language; D009862:Online Systems; D010353:Patient Education as Topic; D011369:Professional-Patient Relations; D014584:User-Computer Interface