Health services research evaluation principles. Broadening a general framework for evaluating health information technology.
BACKGROUND
Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned.


OBJECTIVE
Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks.


METHODS
A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR.


RESULTS
The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged.


CONCLUSIONS
We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.
Drexel University, Mail Stop 1030, Bellet Bldg Room 512, 245 N 15th St., Philadelphia, PA 19027, USA. pss44@drexel.edu
P S
PS
Sockolow
P R
PR
Crawford
H P
HP
Lehmann
D003625:Data Collection; D006302:Health Services Research / Q000379:methods*; D006801:Humans; D008490:Medical Informatics / Q000295:instrumentation / Q000706:statistics & numerical data*; D017146:Models, Organizational; D017060:Patient Satisfaction; D014481:United States