title: How good are the data? Feasible approach to validation of metrics of quality derived from an outpatient electronic health record.
abstract: Although electronic health records (EHRs) promise to be efficient resources for measuring metrics of quality, they are not designed for such population-based analyses. Thus, extracting meaningful clinical data from them is not straightforward. To avoid poorly executed measurements, standardized methods to measure and to validate metrics of quality are needed. This study provides and evaluates a use case for a generally applicable approach to validating quality metrics measured electronically from EHR-based data. The authors iteratively refined and validated 4 outpatient quality metrics and classified errors in measurement. Multiple iterations of validation and measurement resulted in high levels of sensitivity and agreement versus the "gold standard" of manual review. In contrast, substantial differences remained for measurement based on coded billing data. Measuring quality metrics using an EHR-based electronic process requires validation to ensure accuracy; approaches to validation such as those described in this study should be used by organizations measuring quality from EHR-based information.
authors.affiliation: Yale School of Medicine, New Haven, CT 06519, USA. andrea.benin@ynhh.org
authors.forename: Andrea L
authors.initials: AL
authors.lastname: Benin
authors.forename: Ada
authors.initials: A
authors.lastname: Fenick
authors.forename: Jeph
authors.initials: J
authors.lastname: Herrin
authors.forename: Grace
authors.initials: G
authors.lastname: Vitkauskas
authors.forename: John
authors.initials: J
authors.lastname: Chen
authors.forename: Cynthia
authors.initials: C
authors.lastname: Brandt
mesh_terms: D000293:Adolescent; D000553:Ambulatory Care / Q000706:statistics & numerical data; D002648:Child; D002675:Child, Preschool; D019983:Guideline Adherence; D006801:Humans; D007223:Infant; D008403:Mass Screening / Q000458:organization & administration; D016347:Medical Records Systems, Computerized / Q000706:statistics & numerical data*; D010043:Outcome and Process Assessment, Health Care / Q000458:organization & administration*; D010045:Outpatients / Q000706:statistics & numerical data; D061214:Patient Safety; D010372:Pediatrics / Q000458:organization & administration*; D017410:Practice Guidelines as Topic; D019984:Quality Indicators, Health Care / Q000706:statistics & numerical data*; D011787:Quality of Health Care / Q000706:statistics & numerical data; D015203:Reproducibility of Results