title: Quality metrics for detailed clinical models.
abstract: OBJECTIVE
  To develop quality metrics for detailed clinical models (DCMs) and test their validity.
  
  
  METHODS
  Based on existing quality criteria which did not include formal metrics, we developed quality metrics by applying the ISO/IEC 9126 software quality evaluation model. The face and content validity of the initial quality metrics were assessed by 9 international experts. Content validity was defined as agreement by over 70% of the panelists. For eliciting opinions and achieving consensus of the panelists, a two round Delphi survey was conducted. Valid quality metrics were considered reliable if agreement between two evaluators' assessments of two example DCMs was over 0.60 in terms of the kappa coefficient. After reliability and validity were tested, the final DCM quality metrics were selected.
  
  
  RESULTS
  According to the results of the reliability test, the degree of agreement was high (a kappa coefficient of 0.73). Based on the results of the reliability test, 8 quality evaluation domains and 29 quality metrics were finalized as DCM quality metrics.
  
  
  CONCLUSIONS
  Quality metrics were validated by a panel of international DCM experts. Therefore, we expect that the metrics, which constitute essential qualitative and quantitative quality requirements for DCMs, can be used to support rational decision-making by DCM developers and clinical users.
authors.affiliation: Department of Health Policy and Management, College of Medicine in Seoul National University, 28 Yeongeon-dong, Jongno-gu, Seoul, Republic of Korea.
authors.forename: SunJu
authors.initials: S
authors.lastname: Ahn
authors.forename: Stanley M
authors.initials: SM
authors.lastname: Huff
authors.forename: Yoon
authors.initials: Y
authors.lastname: Kim
authors.forename: Dipak
authors.initials: D
authors.lastname: Kalra
mesh_terms: D003695:Delivery of Health Care / Q000592:standards*; D057286:Electronic Health Records / Q000458:organization & administration*; D006296:Health Services / Q000592:standards*; D006801:Humans; D016347:Medical Records Systems, Computerized; D008962:Models, Theoretical; D019984:Quality Indicators, Health Care / Q000458:organization & administration / Q000592:standards*