title: Evaluating the Good Ontology Design Guideline (GoodOD) with the ontology quality requirements and evaluation method and metrics (OQuaRE).
abstract: OBJECTIVE
  To (1) evaluate the GoodOD guideline for ontology development by applying the OQuaRE evaluation method and metrics to the ontology artefacts that were produced by students in a randomized controlled trial, and (2) informally compare the OQuaRE evaluation method with gold standard and competency questions based evaluation methods, respectively.
  
  
  BACKGROUND
  In the last decades many methods for ontology construction and ontology evaluation have been proposed. However, none of them has become a standard and there is no empirical evidence of comparative evaluation of such methods. This paper brings together GoodOD and OQuaRE. GoodOD is a guideline for developing robust ontologies. It was previously evaluated in a randomized controlled trial employing metrics based on gold standard ontologies and competency questions as outcome parameters. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies and has been successfully used for evaluating the quality of ontologies.
  
  
  METHODS
  In this paper, we evaluate the effect of training in ontology construction based on the GoodOD guideline within the OQuaRE quality evaluation framework and compare the results with those obtained for the previous studies based on the same data.
  
  
  RESULTS
  Our results show a significant effect of the GoodOD training over developed ontologies by topics: (a) a highly significant effect was detected in three topics from the analysis of the ontologies of untrained and trained students; (b) both positive and negative training effects with respect to the gold standard were found for five topics.
  
  
  CONCLUSIONS
  The GoodOD guideline had a significant effect over the quality of the ontologies developed. Our results show that GoodOD ontologies can be effectively evaluated using OQuaRE and that OQuaRE is able to provide additional useful information about the quality of the GoodOD ontologies.
authors.affiliation: Departamento de Informática y Sistemas, Facultad de Informática, Universidad de Murcia, Murcia, Spain.
authors.forename: Astrid
authors.initials: A
authors.lastname: Duque-Ramos
authors.affiliation: Department of Medical Biometry and Medical Informatics, University of Freiburg, Freiburg, Germany.
authors.forename: Martin
authors.initials: M
authors.lastname: Boeker
authors.affiliation: Department of Philosophy, University of Muenster, Muenster, Germany.
authors.forename: Ludger
authors.initials: L
authors.lastname: Jansen
authors.affiliation: Institute of Medical Informatics, Statistics and Documentation Medical University of Graz, Graz, Austria.
authors.forename: Stefan
authors.initials: S
authors.lastname: Schulz
authors.affiliation: Departamento de Estadística e Investigación Operativa, Universidad de Murcia, Murcia, Spain.
authors.forename: Miguela
authors.initials: M
authors.lastname: Iniesta
authors.affiliation: Departamento de Informática y Sistemas, Facultad de Informática, Universidad de Murcia, Murcia, Spain.
authors.forename: Jesualdo Tomás
authors.initials: JT
authors.lastname: Fernández-Breis
mesh_terms: D017408:Guidelines as Topic / Q000592:standards*; D006801:Humans; D008962:Models, Theoretical; D018875:Vocabulary, Controlled