'Televaluation' of clinical information systems: an integrative approach to assessing Web-based systems.
The World Wide Web provides an unprecedented opportunity for widespread access to health-care applications by both patients and providers. The development of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance evaluation (i.e. 'televaluation') of emerging Web-based information technologies. In health informatics evaluation, there is a need for application of new ideas and methods from the fields of cognitive science and usability engineering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video-based usability testing of user interactions with clinical information systems. Examples illustrating application of these techniques are presented for the assessment of a patient clinical information system (PatCIS), as well as an evaluation of use of Web-based clinical guidelines. Issues in designing, prototyping and iteratively refining evaluation components are discussed, along with description of a 'virtual' usability laboratory.
Kushniruk
A W
AW
Department of Mathematics and Statistics, York University, 4700 Keele Street, Ontario, M3J 1P3, Toronto, Canada. andrek@mathstat.yorku.ca
Patel
C
C
Patel
V L
VL
Cimino
J J
JJ
D001292:Attitude to Computers; D005069:Evaluation Studies as Topic; D006801:Humans; D007255:Information Services; D020407:Internet; D010353:Patient Education as Topic; D017060:Patient Satisfaction; D011795:Surveys and Questionnaires; D017216:Telemedicine; D014584:User-Computer Interface