Integrating SNOMED CT into the UMLS: an exploration of different views of synonymy and quality of editing.
OBJECTIVE
The integration of SNOMED CT into the Unified Medical Language System (UMLS) involved the alignment of two views of synonymy that were different because the two vocabulary systems have different intended purposes and editing principles. The UMLS is organized according to one view of synonymy, but its structure also represents all the individual views of synonymy present in its source vocabularies. Despite progress in knowledge-based automation of development and maintenance of vocabularies, manual curation is still the main method of determining synonymy. The aim of this study was to investigate the quality of human judgment of synonymy.


METHODS
Sixty pairs of potentially controversial SNOMED CT synonyms were reviewed by 11 domain vocabulary experts (six UMLS editors and five noneditors), and scores were assigned according to the degree of synonymy.


METHODS
The synonymy scores of each subject were compared to the gold standard (the overall mean synonymy score of all subjects) to assess accuracy. Agreement between UMLS editors and noneditors was measured by comparing the mean synonymy scores of editors to noneditors.


RESULTS
Average accuracy was 71% for UMLS editors and 75% for noneditors (difference not statistically significant). Mean scores of editors and noneditors showed significant positive correlation (Spearman's rank correlation coefficient 0.654, two-tailed p < 0.01) with a concurrence rate of 75% and an interrater agreement kappa of 0.43.


CONCLUSIONS
The accuracy in the judgment of synonymy was comparable for UMLS editors and nonediting domain experts. There was reasonable agreement between the two groups.
Fung
Kin Wah
KW
National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA. kwfung@nlm.nih.gov
Hole
William T
WT
Nelson
Stuart J
SJ
Srinivasan
Suresh
S
Powell
Tammy
T
Roth
Laura
L
D012660:Semantics; D039061:Systematized Nomenclature of Medicine; D018511:Systems Integration; D017432:Unified Medical Language System / Q000458:organization & administration; D018875:Vocabulary, Controlled