title: Evaluating alignment quality between iconic language and reference terminologies using similarity metrics.
abstract: BACKGROUND
  Visualization of Concepts in Medicine (VCM) is a compositional iconic language that aims to ease information retrieval in Electronic Health Records (EHR), clinical guidelines or other medical documents. Using VCM language in medical applications requires alignment with medical reference terminologies. Alignment from Medical Subject Headings (MeSH) thesaurus and International Classification of Diseases - tenth revision (ICD10) to VCM are presented here. This study aim was to evaluate alignment quality between VCM and other terminologies using different measures of inter-alignment agreement before integration in EHR.
  
  
  METHODS
  For medical literature retrieval purposes and EHR browsing, the MeSH thesaurus and the ICD10, both organized hierarchically, were aligned to VCM language. Some MeSH to VCM alignments were performed automatically but others were performed manually and validated. ICD10 to VCM alignment was entirely manually performed. Inter-alignment agreement was assessed on ICD10 codes and MeSH descriptors, sharing the same Concept Unique Identifiers in the Unified Medical Language System (UMLS). Three metrics were used to compare two VCM icons: binary comparison, crude Dice Similarity Coefficient (DSCcrude), and semantic Dice Similarity Coefficient (DSCsemantic), based on Lin similarity. An analysis of discrepancies was performed.
  
  
  RESULTS
  MeSH to VCM alignment resulted in 10,783 relations: 1,830 of which were manually performed and 8,953 were automatically inherited. ICD10 to VCM alignment led to 19,852 relations. UMLS gathered 1,887 alignments between ICD10 and MeSH. Only 1,606 of them were used for this study. Inter-alignment agreement using only validated MeSH to VCM alignment was 74.2% [70.5-78.0]CI95%, DSCcrude was 0.93 [0.91-0.94]CI95%, and DSCsemantic was 0.96 [0.95-0.96]CI95%. Discrepancy analysis revealed that even if two thirds of errors came from the reviewers, UMLS was nevertheless responsible for one third.
  
  
  CONCLUSIONS
  This study has shown strong overall inter-alignment agreement between MeSH to VCM and ICD10 to VCM manual alignments. VCM icons have now been integrated into a guideline search engine (http://www.cismef.org) and a health terminologies portal (http://www.hetop.eu).
authors.affiliation: CISMeF, Rouen University Hospital, Normandy & TIBS, LITIS EA 4108, Institute for Research and Innovation in Biomedicine, Rouen, France. nicolas.griffon@chu-rouen.fr.
authors.forename: Nicolas
authors.initials: N
authors.lastname: Griffon
authors.forename: Gaetan
authors.initials: G
authors.lastname: Kerdelhué
authors.forename: Lina F
authors.initials: LF
authors.lastname: Soualmia
authors.forename: Tayeb
authors.initials: T
authors.lastname: Merabti
authors.forename: Julien
authors.initials: J
authors.lastname: Grosjean
authors.forename: Jean-Baptiste
authors.initials: JB
authors.lastname: Lamy
authors.forename: Alain
authors.initials: A
authors.lastname: Venot
authors.forename: Catherine
authors.initials: C
authors.lastname: Duclos
authors.forename: Stefan J
authors.initials: SJ
authors.lastname: Darmoni
mesh_terms: D057286:Electronic Health Records / Q000592:standards; D006801:Humans; D016247:Information Storage and Retrieval / Q000592:standards*; D038801:International Classification of Diseases / Q000706:statistics & numerical data; D046650:Medical Subject Headings / Q000706:statistics & numerical data; D009626:Terminology as Topic; D017432:Unified Medical Language System / Q000592:standards; D018875:Vocabulary, Controlled