Using computer-extracted data from electronic health records to measure the quality of adolescent well-care.
OBJECTIVE
To determine whether quality measures based on computer-extracted EHR data can reproduce findings based on data manually extracted by reviewers.


METHODS
We studied 12 measures of care indicated for adolescent well-care visits for 597 patients in three pediatric health systems.


METHODS
Observational study.


METHODS
Manual reviewers collected quality data from the EHR. Site personnel programmed their EHR systems to extract the same data from structured fields in the EHR according to national health IT standards.


RESULTS
Overall performance measured via computer-extracted data was 21.9 percent, compared with 53.2 percent for manual data. Agreement measures were high for immunizations. Otherwise, agreement between computer extraction and manual review was modest (Kappa = 0.36) because computer-extracted data frequently missed care events (sensitivity = 39.5 percent). Measure validity varied by health care domain and setting. A limitation of our findings is that we studied only three domains and three sites.


CONCLUSIONS
The accuracy of computer-extracted EHR quality reporting depends on the use of structured data fields, with the highest agreement found for measures and in the setting that had the greatest concentration of structured fields. We need to improve documentation of care, data extraction, and adaptation of EHR systems to practice workflow.
Department of Pediatrics, Dalhousie University, Halifax, NS.
William
W
Gardner
Suzanne
S
Morton
Sepheen C
SC
Byron
Aldo
A
Tinoco
Benjamin D
BD
Canan
Karen
K
Leonhart
Vivian
V
Kong
Sarah Hudson
SH
Scholle
D000293:Adolescent; D016497:Adolescent Health Services / Q000592:standards*; D002648:Child; D003201:Computers; D003625:Data Collection / Q000379:methods; D057225:Data Mining; D037001:Directive Counseling; D057286:Electronic Health Records; D005260:Female; D006801:Humans; D008297:Male; D019984:Quality Indicators, Health Care; D055815:Young Adult
Quality measurement; electronic health records; pediatric well-care