title: Measuring data reliability for preventive services in electronic medical records.
abstract: BACKGROUND
  Improvements in the quality of health care services are often measured using data present in medical records. Electronic Medical Records (EMRs) contain potentially valuable new sources of health data. However, data quality in EMRs may not be optimal and should be assessed. Data reliability (are the same data elements being measured over time?) is a prerequisite for data validity (are the data accurate?). Our objective was to measure the reliability of data for preventive services in primary care EMRs during the transition to EMR.
  
  
  METHODS
  Our data sources were randomly selected eligible patients' medical records and data obtained from provincial administrative datasets. Eighteen community-based family physicians in Toronto, Ontario that implemented EMRs starting in 2006 participated in this study. We measured the proportion of patients eligible for a service (Pap smear, screening mammogram or influenza vaccination) that received the service. We compared the change in rates of selected preventive services calculated from the medical record audits with the change in administrative datasets.
  
  
  RESULTS
  In the first year of EMR use (2006) services decreased by 8.7% more (95% CI -11.0%- - 6.4%, p < 0.0001) when measured through medical record audits as compared with administrative datasets. Services increased by 2.4% more (95% CI 0%-4.9%, p = 0.05) in the medical record audits during the second year of EMR use (2007).
  
  
  CONCLUSIONS
  There were differences between the change measured through medical record audits and administrative datasets. Problems could include difficulties with organizing new data entry processes as well as continued use of both paper and EMRs. Data extracted from EMRs had limited reliability during the initial phase of EMR implementation. Unreliable data interferes with the ability to measure and improve health care quality.
authors.affiliation: Department of Family and Community Medicine, University of Toronto, Duncan Mill Road, Suite 705, Toronto, ON, Canada. mgreiver@rogers.com
authors.forename: Michelle
authors.initials: M
authors.lastname: Greiver
authors.forename: Jan
authors.initials: J
authors.lastname: Barnsley
authors.forename: Richard H
authors.initials: RH
authors.lastname: Glazier
authors.forename: Bart J
authors.initials: BJ
authors.lastname: Harvey
authors.forename: Rahim
authors.initials: R
authors.lastname: Moineddin
mesh_terms: D000328:Adult; D002908:Chronic Disease / Q000453:epidemiology / Q000517:prevention & control; D015897:Comorbidity; D016208:Databases, Factual; D003953:Diagnosis-Related Groups; D003955:Diagnostic Tests, Routine / Q000592:standards / Q000706:statistics & numerical data; D057286:Electronic Health Records / Q000592:standards* / Q000706:statistics & numerical data; D005260:Female; D006302:Health Services Research; D006801:Humans; D008485:Medical Audit / Q000706:statistics & numerical data; D008499:Medical Records / Q000592:standards*; D008875:Middle Aged; D009864:Ontario; D010821:Physicians, Family / Q000458:organization & administration / Q000592:standards*; D010818:Practice Patterns, Physicians' / Q000706:statistics & numerical data; D011314:Preventive Health Services / Q000592:standards* / Q000706:statistics & numerical data; D019984:Quality Indicators, Health Care; D011787:Quality of Health Care / Q000191:economics / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D013996:Time and Motion Studies