title: From an expert-driven paper guideline to a user-centred decision support system: a usability comparison study.
abstract: OBJECTIVE
  To assess whether a user-centred prototype clinical decision support system (CDSS) providing patient-specific advice better supports healthcare practitioners in terms of (a) types of usability problems detected and (b) effective and efficient retrieval of childhood cancer survivor's follow-up screening procedures compared to an expert-driven paper-based guideline.
  
  
  METHODS
  A user-centred design (UCD) process was employed to design a prototype CDSS. Usability problems in information retrieval with the paper-based guideline were assessed by think-aloud analysis with 13 participants. Both simple and more complex tasks were applied. The analysis provided input for the UCD process of the prototype. The usability of the prototype CDSS was subsequently evaluated by think-aloud analysis with the same participants. Usability problems of the paper-based guideline and the prototype CDSS were compared by using the classification of usability problems scheme. In addition, efficiency (time to complete task) and effectiveness (completeness of retrieved screening procedures) of information retrieval of participants in the expert-driven paper-based guideline and the user-centred prototype CDSS were compared.
  
  
  RESULTS
  Usability problems in both the paper-based guideline and the CDSS prototype were mainly classified as 'incongruent with participants' mental model'. The prototype CDSS reduced this type of problem from 17 to 6 problems. The time to perform simple information retrieval tasks increased by 58 s when using the prototype CDSS, however, it resulted in a 58% improvement in task completeness compared to the paper-based guideline. The time to perform complex scenarios decreased by 3:50 min with the prototype CDSS, with 17% higher completeness compared to the paper-based guideline.
  
  
  CONCLUSIONS
  Analysis showed that usability problems experienced by healthcare practitioners when using a paper-based guideline could be overcome by implementing the guideline in a user-centred CDSS design. Although different types of usability problems were experienced with the prototype CDSS, they did not inhibit effective and efficient performance of tasks in the system. The usability problem analysis of the paper-based guideline effectively supported comparison of usability problems found in the two information retrieval systems and it supported the UCD of the CDSS.
authors.affiliation: Department of Medical Informatics, Centre for Human Factors Engineering of Interactive Health Information Technology, Academic Medical Center - University of Amsterdam, PO Box 22660, 1100 DD Amsterdam, The Netherlands. Electronic address: e.kilsdonk@amc.uva.nl.
authors.forename: Ellen
authors.initials: E
authors.lastname: Kilsdonk
authors.forename: Linda W
authors.initials: LW
authors.lastname: Peute
authors.forename: Rinke J
authors.initials: RJ
authors.lastname: Riezebos
authors.forename: Leontien C
authors.initials: LC
authors.lastname: Kremer
authors.forename: Monique W M
authors.initials: MW
authors.lastname: Jaspers
mesh_terms: D020000:Decision Support Systems, Clinical; D017408:Guidelines as Topic; D017063:Outcome Assessment, Health Care
keywords: Childhood cancer survivors; Clinical decision support system; Screening guidelines; Usability analysis; User-centred design