title: Automating care quality measurement with health information technology.
abstract: OBJECTIVE
  To assess the performance of a health information technology platform that enables automated measurement of asthma care quality using comprehensive electronic medical record (EMR) data, including providers' free-text notes.
  
  
  METHODS
  Retrospective data study of outpatient asthma care in Kaiser Permanente Northwest (KPNW), a midsized health maintenance organization (HMO), and OCHIN, Inc, a group of Federally Qualified Health Centers.
  
  
  METHODS
  We created 22 automated quality measures addressing guideline-recommended outpatient asthma care. We included EMRs of asthma patients aged >12 years during a 3-year observation window and narrowed this group to those with persistent asthma (13,918 KPNW; 1825 OCHIN). We validated our automated quality measures using chart review for 818 randomly selected patients, stratified by age and sex for each health system. In both health systems, we compared the performance of these measures against chart review.
  
  
  RESULTS
  Most measures performed well in the KPNW system, where accuracy averaged 88% (95% confidence interval [CI] 82%-93%). Mean sensitivity was 77% (95% CI 62%-92%) and mean specificity was 84% (95% CI 75%-93%). The automated analysis was less accurate at OCHIN, where mean accuracy was 80% (95% CI 72%-89%) with mean sensitivity and specificity 52% (95% CI 35%-69%) and 82% (95% CI 69%-95%) respectively.
  
  
  CONCLUSIONS
  To achieve comprehensive quality measurement in many clinical domains, the capacity to analyze text clinical notes is required. The automated measures performed well in the HMO, where practice is more standardized. The measures need to be refined for health systems with more diversity in clinical practice, patient populations, and setting.
authors: Center for Health Research, Kaiser Permanente Northwest, Portland, OR 97227, USA. Brian.Hazlehurst@kpchr.org
authors: Brian
authors: B
authors: Hazelhurst
authors: Mary Ann
authors: MA
authors: McBurnie
authors: Richard A
authors: RA
authors: Mularski
authors: Jon E
authors: JE
authors: Puro
authors: Susan L
authors: SL
authors: Chauvie
mesh_terms: D018927:Anti-Asthmatic Agents / Q000627:therapeutic use*; D001249:Asthma / Q000188:drug therapy*; D001331:Automation; D019985:Benchmarking; D016001:Confidence Intervals; D057286:Electronic Health Records; D006801:Humans; D008490:Medical Informatics / Q000379:methods*; D010045:Outpatients; D011787:Quality of Health Care / Q000706:statistics & numerical data*; D012189:Retrospective Studies; D014481:United States
chemical_list: D018927:Anti-Asthmatic Agents