title: Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data.
abstract: OBJECTIVE
  Validation of health administrative data for identifying patients with different health states (diseases and conditions) is a research priority, but no guidelines exist for ensuring quality. We created reporting guidelines for studies validating administrative data identification algorithms and used them to assess the quality of reporting of validation studies in the literature.
  
  
  METHODS
  Using Standards for Reporting of Diagnostic accuracy (STARD) criteria as a guide, we created a 40-item checklist of items with which identification accuracy studies should be reported. A systematic review identified studies that validated identification algorithms using administrative data. We used the checklist to assess the quality of reporting.
  
  
  RESULTS
  In 271 included articles, goals and data sources were well reported but few reported four or more statistical estimates of accuracy (36.9%). In 65.9% of studies reporting positive predictive value (PPV)/negative predictive value (NPV), the prevalence of disease in the validation cohort was higher than in the administrative data, potentially falsely elevating predictive values. Subgroup accuracy (53.1%) and 95% confidence intervals for accuracy measures (35.8%) were also underreported.
  
  
  CONCLUSIONS
  The quality of studies validating health states in the administrative data varies, with significant deficits in reporting of markers of diagnostic accuracy, including the appropriate estimation of PPV and NPV. These omissions could lead to misclassification bias and incorrect estimation of incidence and health services utilization rates. Use of a reporting checklist, such as the one created for this study by modifying the STARD criteria, could improve the quality of reporting of validation studies, allowing for accurate application of algorithms, and interpretation of research using health administrative data.
authors.affiliation: The Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada. ebenchimol@cheo.on.ca
authors.forename: Eric I
authors.initials: EI
authors.lastname: Benchimol
authors.forename: Douglas G
authors.initials: DG
authors.lastname: Manuel
authors.forename: Teresa
authors.initials: T
authors.lastname: To
authors.forename: Anne M
authors.initials: AM
authors.lastname: Griffiths
authors.forename: Linda
authors.initials: L
authors.lastname: Rabeneck
authors.forename: Astrid
authors.initials: A
authors.lastname: Guttmann
mesh_terms: D000465:Algorithms; D057189:Checklist / Q000592:standards*; D005568:Forms and Records Control / Q000592:standards; D017408:Guidelines as Topic; D006302:Health Services Research / Q000592:standards*; D006304:Health Status; D006753:Hospital Records / Q000592:standards*; D006801:Humans; D038801:International Classification of Diseases; D011786:Quality Control; D057170:Translational Research, Biomedical; D054928:Validation Studies as Topic