title: Evaluating semantic similarity and relatedness over the semantic grouping of clinical term pairs.
abstract: BACKGROUND
  This article explores how measures of semantic similarity and relatedness are impacted by the semantic groups to which the concepts they are measuring belong. Our goal is to determine if there are distinctions between homogeneous comparisons (where both concepts belong to the same group) and heterogeneous ones (where the concepts are in different groups). Our hypothesis is that the similarity measures will be significantly affected since they rely on hierarchical is-a relations, whereas relatedness measures should be less impacted since they utilize a wider range of relations. In addition, we also evaluate the effect of combining different measures of similarity and relatedness. Our hypothesis is that these combined measures will more closely correlate with human judgment, since they better reflect the rich variety of information humans use when assessing similarity and relatedness.
  
  
  METHODS
  We evaluate our method on four reference standards. Three of the reference standards were annotated by human judges for relatedness and one was annotated for similarity.
  
  
  RESULTS
  We found significant differences in the correlation of semantic similarity and relatedness measures with human judgment, depending on which semantic groups were involved. We also found that combining a definition based relatedness measure with an information content similarity measure resulted in significant improvements in correlation over individual measures.
  
  
  BACKGROUND
  The semantic similarity and relatedness package is an open source program available from http://umls-similarity.sourceforge.net/. The reference standards are available at http://www.people.vcu.edu/∼{}btmcinnes/downloads.html.
authors.affiliation: Department of Computer Science, Virginia Commonwealth University, 401 S. Main St., Rm E4225, Richmond, VA 23284, USA. Electronic address: btmcinnes@vcu.edu.
authors.forename: Bridget T
authors.initials: BT
authors.lastname: McInnes
authors.affiliation: Department of Computer Science, University of Minnesota, 1114 Kirby Drive, Duluth, MN 55812, USA. Electronic address: tpederse@d.umn.edu.
authors.forename: Ted
authors.initials: T
authors.lastname: Pedersen
mesh_terms: D006801:Humans; D009323:Natural Language Processing; D012660:Semantics; D039061:Systematized Nomenclature of Medicine; D017432:Unified Medical Language System / Q000145:classification*
keywords: NLP; Natural language processing; Semantic relatedness; Semantic similarity