Machine vs. human translation of SNOMED CT terms.
OBJECTIVE
In the context of past and current SNOMED CT translation projects we compare three kinds of SNOMED CT translations from English to German by: (t1) professional medical translators; (t2) a free Web-based machine translation service; (t3) medical students.


METHODS
500 SNOMED CT fully specified names from the (English) International release were randomly selected. Based on this, German translations t1, t2, and t3 were generated. A German and an Austrian physician rated the translations for linguistic correctness and content fidelity.


RESULTS
Kappa for inter-rater reliability was 0.4 for linguistic correctness and 0.23 for content fidelity. Average ratings of linguistic correctness did not differ significantly between human translation scenarios. Content fidelity was rated slightly better for student translators compared to professional translators. Comparing machine to human translation, the linguistic correctness differed about 0.5 scale units in favour of the human translation and about 0.25 regarding content fidelity, equally in favour of the human translation.


CONCLUSIONS
The results demonstrate that low-cost translation solutions of medical terms may produce surprisingly good results. Although we would not recommend low-cost translation for producing standardized preferred terms, this approach can be useful for creating additional language-specific entry terms. This may serve several important use cases. We also recommend testing this method to bootstrap a crowdsourcing process, by which term translations are gathered, improved, maintained, and rated by the user community.
Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Austria.
Stefan
S
Schulz
Johannes
J
Bernhardt-Melischnig
Markus
M
Kreuzthaler
Philipp
P
Daumke
Martin
M
Boeker
D001185:Artificial Intelligence; D005858:Germany; D006801:Humans; D020407:Internet; D009323:Natural Language Processing; D012984:Software; D012986:Software Validation; D039061:Systematized Nomenclature of Medicine; D009626:Terminology as Topic; D014175:Translating; D014481:United States