title: Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies.
abstract: OBJECTIVE
  Standards terminologies may be large and complex, making their quality assurance challenging. Some terminology quality assurance (TQA) methodologies are based on abstraction networks (AbNs), compact terminology summaries. We have tested AbNs and the performance of related TQA methodologies on small terminology hierarchies. However, some standards terminologies, for example, SNOMED, are composed of very large hierarchies. Scaling AbN TQA techniques to such hierarchies poses a significant challenge. We present a scalable subject-based approach for AbN TQA.
  
  
  METHODS
  An innovative technique is presented for scaling TQA by creating a new kind of subject-based AbN called a subtaxonomy for large hierarchies. New hypotheses about concentrations of erroneous concepts within the AbN are introduced to guide scalable TQA.
  
  
  RESULTS
  We test the TQA methodology for a subject-based subtaxonomy for the Bleeding subhierarchy in SNOMED's large Clinical finding hierarchy. To test the error concentration hypotheses, three domain experts reviewed a sample of 300 concepts. A consensus-based evaluation identified 87 erroneous concepts. The subtaxonomy-based TQA methodology was shown to uncover statistically significantly more erroneous concepts when compared to a control sample.
  
  
  CONCLUSIONS
  The scalability of TQA methodologies is a challenge for large standards systems like SNOMED. We demonstrated innovative subject-based TQA techniques by identifying groups of concepts with a higher likelihood of having errors within the subtaxonomy. Scalability is achieved by reviewing a large hierarchy by subject.
  
  
  CONCLUSIONS
  An innovative methodology for scaling the derivation of AbNs and a TQA methodology was shown to perform successfully for the largest hierarchy of SNOMED.
authors.affiliation: Computer Science Department, New Jersey Institute of Technology, Newark, New Jersey, USA.
authors.forename: Christopher
authors.initials: C
authors.lastname: Ochs
authors.affiliation: Computer Science Department, New Jersey Institute of Technology, Newark, New Jersey, USA.
authors.forename: James
authors.initials: J
authors.lastname: Geller
authors.affiliation: Computer Science Department, New Jersey Institute of Technology, Newark, New Jersey, USA.
authors.forename: Yehoshua
authors.initials: Y
authors.lastname: Perl
authors.affiliation: Computer Information Systems Department, BMCC, CUNY, New York, New York, USA.
authors.forename: Yan
authors.initials: Y
authors.lastname: Chen
authors.affiliation: Division of Knowledge Informatics, NYU, New York, New York, USA.
authors.forename: Junchuan
authors.initials: J
authors.lastname: Xu
authors.affiliation: Department of Health Administration and Policy, George Mason University, Fairfax, Virginia, USA.
authors.forename: Hua
authors.initials: H
authors.lastname: Min
authors.affiliation: NLM/NIH, Bethesda, Maryland, USA.
authors.forename: James T
authors.initials: JT
authors.lastname: Case
authors.affiliation: Computer Science Department, New Jersey Institute of Technology, Newark, New Jersey, USA.
authors.forename: Zhi
authors.initials: Z
authors.lastname: Wei
mesh_terms: D002965:Classification; D006470:Hemorrhage / Q000145:classification*; D006801:Humans; D009369:Neoplasms / Q000145:classification*; D011786:Quality Control; D039061:Systematized Nomenclature of Medicine
keywords: SNOMED CT; abstraction network; scalable quality assurance; standards quality assurance; subject-based terminology quality assurance; terminology quality assurance