title: Evaluating the Reliability of EHR-Generated Clinical Outcomes Reports: A Case Study.
abstract: BACKGROUND
  Quality incentive programs, such as Meaningful Use, operate under the assumption that clinical quality measures can be reliably extracted from EHRs. Safety Net providers, particularly Federally Qualified Health Centers and Look-Alikes, tend to be high adopters of EHRs; however, recent reports have shown that only about 9% of FQHCs and Look-Alikes were demonstrating meaningful use as of 2013. Our experience working with the Crescent City Beacon Community (CCBC) found that many health centers relied on chart audits to report quality measures as opposed to electronically generating reports directly from their EHRs due to distrust in the data. This paper describes a step-by-step process for improving the reliability of data extracted from EHRs to increase reliability of quality measure reports, to support quality improvement, and to achieve alignment with national clinical quality reporting requirements.
  
  
  BACKGROUND
  Lack of standardization in data capture and reporting within EHRs drives distrust in EHR-reported data. Practices or communities attempting to achieve standardization may look to CCBC's experience for guidance on where to start and the level of resources required in order to execute a data standardization project. During the time of this data standardization project, CCBC was launching an HIE. Lack of trust in EHR data was a driver for distrust in the HIE data.
  
  
  METHODS
  We present a case study where a five-step process was used to harmonize measures, reduce data errors, and increase trust in EHR clinical outcomes reports among a community of Safety Net providers using a common EHR. Primary outcomes were the incidence of reporting errors and the potential effect of error types on quality measure percentages. The activities and level of resources required to achieve these results were also documented by the CCBC program.
  
  
  RESULTS
  Implementation of a community-wide data reporting project resulted in measure harmonization, reduced reporting burden, and error reduction in EHR-generated clinical outcomes reporting across participating clinics over a nine-month period. Increased accuracy of clinical outcomes reports provided physicians and clinical care teams with better information to guide their decision-making around quality improvement planning.
  
  
  CONCLUSIONS
  A number of challenges exist to achieving reliable population level quality reporting from EHRs at the practice, vendor, and community levels. Our experience demonstrates that quality measure reporting from EHRs is not a straightforward process, and it requires time and close collaboration between clinics and vendors to improve reliability of reports. Our experience found that practices valued the opportunity and step-wise process to validate their data locally (out of their EHRs) prior to reporting out of the HIE.
  Communities can achieve higher levels of confidence in quality measure reporting at the population level by establishing collaborative user groups that work with EHR vendors as partners and use technical assistance to build relationships and trust in EHR-generated reports. While this paper describes the first phase of our work around improving standardization and reliability of EHR reports, vendors should continue to explore modifications for improving data capture (at the front-end) via standardized data entry templates.
authors.affiliation: Louisiana Public Health Institute.
authors.forename: Chatrian
authors.initials: C
authors.lastname: Kanger
authors.affiliation: Louisiana Public Health Institute.
authors.forename: Lisanne
authors.initials: L
authors.lastname: Brown
authors.affiliation: Louisiana Public Health Institute.
authors.forename: Snigdha
authors.initials: S
authors.lastname: Mukherjee
authors.affiliation: University of Alabama at Birmingham.
authors.forename: Haichang
authors.initials: H
authors.lastname: Xin
authors.affiliation: Tulane University.
authors.forename: Mark L
authors.initials: ML
authors.lastname: Diana
authors.affiliation: Louisiana Public Health Institute.
authors.forename: Anjum
authors.initials: A
authors.lastname: Khurshid
keywords: data use and quality; health information technology; standardized data collection
references.citation: J Am Med Inform Assoc. 2012 Jul-Aug;19(4):610-4
references.pmid: 22199017
references.citation: Int J Qual Health Care. 2012 Jun;24(3):200-5
references.pmid: 22490301
references.citation: J Healthc Qual. 2016 Jan-Feb;38(1):42-51
references.pmid: 24612263
references.citation: EGEMS (Wash DC). 2014 Aug 04;2(3):1094
references.pmid: 25848621
references.citation: Ann Intern Med. 2013 Jan 15;158(2):77-83
references.pmid: 23318309
references.citation: J Am Med Inform Assoc. 2012 Jun;19(e1):e125-8
references.pmid: 22323393
references.citation: N Engl J Med. 2010 Aug 5;363(6):501-4
references.pmid: 20647183
references.citation: Med Care Res Rev. 2010 Oct;67(5):503-27
references.pmid: 20150441