Future directions in evaluation research: people, organizational, and social issues.
OBJECTIVE
To review evaluation literature concerning people, organizational, and social issues and provide recommendations for future research.


METHODS
Analyze this research and make recommendations.


CONCLUSIONS
Evaluation research is key in identifying how people, organizational, and social issues - all crucial to system design, development, implementation, and use - interplay with informatics projects. Building on a long history of contributions and using a variety of methods, researchers continue developing evaluation theories and methods while producing significant interesting studies. We recommend that future research: 1) Address concerns of the many individuals involved in or affected by informatics applications. 2) Conduct studies in different type and size sites, and with different scopes of systems and different groups of users. Do multi-site or multi-system comparative studies. 3) Incorporate evaluation into all phases of a project. 4) Study failures, partial successes, and changes in project definition or outcome. 5) Employ evaluation approaches that take account of the shifting nature of health care and project environments, and do formative evaluations. 6) Incorporate people, social, organizational, cultural, and concomitant ethical issues into the mainstream of medical informatics. 7) Diversify research approaches and continue to develop new approaches. 8) Conduct investigations at different levels of analysis. 9) Integrate findings from different applications and contextual settings, different areas of health care, studies in other disciplines, and also work that is not published in traditional research outlets. 10) Develop and test theory to inform both further evaluation research and informatics practice.
Kaplan
B
B
Kaplan Associates, 59 Morris Street, Hamden, CT 06517, USA. bonnie.kaplan@yale.edu
Shaw
N T
NT
D002170:Canada; D003299:Cooperative Behavior; D003695:Delivery of Health Care / Q000458:organization & administration*; D005069:Evaluation Studies as Topic; D006801:Humans; D008490:Medical Informatics / Q000639:trends*; D013673:Technology Assessment, Biomedical; D017216:Telemedicine; D014481:United States