title: Relating Complexity and Error Rates of Ontology Concepts. More Complex NCIt Concepts Have More Errors.
abstract: OBJECTIVE
  Ontologies are knowledge structures that lend support to many health-information systems. A study is carried out to assess the quality of ontological concepts based on a measure of their complexity. The results show a relation between complexity of concepts and error rates of concepts.
  
  
  METHODS
  A measure of lateral complexity defined as the number of exhibited role types is used to distinguish between more complex and simpler concepts. Using a framework called an area taxonomy, a kind of abstraction network that summarizes the structural organization of an ontology, concepts are divided into two groups along these lines. Various concepts from each group are then subjected to a two-phase QA analysis to uncover and verify errors and inconsistencies in their modeling. A hierarchy of the National Cancer Institute thesaurus (NCIt) is used as our test-bed. A hypothesis pertaining to the expected error rates of the complex and simple concepts is tested.
  
  
  RESULTS
  Our study was done on the NCIt's Biological Process hierarchy. Various errors, including missing roles, incorrect role targets, and incorrectly assigned roles, were discovered and verified in the two phases of our QA analysis. The overall findings confirmed our hypothesis by showing a statistically significant difference between the amounts of errors exhibited by more laterally complex concepts vis-à-vis simpler concepts.
  
  
  CONCLUSIONS
  QA is an essential part of any ontology's maintenance regimen. In this paper, we reported on the results of a QA study targeting two groups of ontology concepts distinguished by their level of complexity, defined in terms of the number of exhibited role types. The study was carried out on a major component of an important ontology, the NCIt. The findings suggest that more complex concepts tend to have a higher error rate than simpler concepts. These findings can be utilized to guide ongoing efforts in ontology QA.
authors.affiliation: Hua Min, Department of Health Administration and Policy, College of Health and Human Services, George Mason University, MS: 1J3, 4400 University Drive, Fairfax, VA 22030-4444, USA, E-mail: hmin3@gmu.edu.
authors.forename: Hua
authors.initials: H
authors.lastname: Min
authors.forename: Ling
authors.initials: L
authors.lastname: Zheng
authors.forename: Yehoshua
authors.initials: Y
authors.lastname: Perl
authors.forename: Michael
authors.initials: M
authors.lastname: Halper
authors.forename: Sherri
authors.initials: S
authors.lastname: De Coronado
authors.forename: Christopher
authors.initials: C
authors.lastname: Ochs
mesh_terms: D064229:Biological Ontologies; D032882:Comprehension; D003198:Computer Simulation; D006801:Humans; D062527:Meaningful Use / Q000592:standards*; D015233:Models, Statistical; D054547:National Cancer Institute (U.S.) / Q000592:standards*; D009323:Natural Language Processing; D009369:Neoplasms / Q000145:classification*; D011785:Quality Assurance, Health Care / Q000592:standards; D014481:United States; D018875:Vocabulary, Controlled
keywords: National Cancer Institute thesaurus; Ontology quality assurance; abstraction network; ontology complexity; ontology modeling