title: Applying an evaluation framework for health information system design, development, and implementation.
abstract: BACKGROUND
  Dramatic advances in health information technologies necessitate a comprehensive evaluation framework covering all phases of development, from conception to routine operational use.
  
  
  OBJECTIVE
  The purpose of this article was to illustrate the application of an informatics evaluation framework that provides a heuristic for matching the stage of system design and the level of evaluation.
  
  
  METHODS
  An evaluation framework is illustrated in the context of five studies in different stages of system design.
  
  
  RESULTS
  The studies discussed in this article represent the various stages of the framework. In addition, they are examples of how evaluation research studies not only contribute to the assessment of system design but also constitute distinct contributions to knowledge.
  
  
  CONCLUSIONS
  The ability to engineer advanced information systems has exceeded the understanding of how to deploy them effectively in complex settings and to adapt them to a range of user populations. A systematic, continuous evaluation increases the likelihood that the conditions for success will be understood, including how to tailor a system, how to maintain the target population's use of the system, and how to innovate continually to enhance the functionality of the system and the users' experience. Without sound evaluation methodologies throughout the stages of system development, information systems have limited potential to influence healthcare processes positively.
authors.affiliation: Department of Biomedical Informatics, NY State Psychiastric Institute, Columbia University School of Nursing, 622 West 168th Street, New York, NY, USA. davek@dbmi.columbia.edu
authors.forename: David
authors.initials: D
authors.lastname: Kaufman
authors.forename: Wm Dan
authors.initials: WD
authors.lastname: Roberts
authors.forename: Jacqueline
authors.initials: J
authors.lastname: Merrill
authors.forename: Tsai-Ya
authors.initials: TY
authors.lastname: Lai
authors.forename: Suzanne
authors.initials: S
authors.lastname: Bakken
mesh_terms: D006801:Humans; D007256:Information Systems / Q000458:organization & administration*; D015403:Nursing Evaluation Research / Q000379:methods; D015397:Program Evaluation / Q000379:methods*; D013597:Systems Analysis; D018511:Systems Integration; D014584:User-Computer Interface; D018875:Vocabulary, Controlled