title: Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.
abstract: Audit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety.
  
  
  
  To evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR).
  
  
  
  (1) Describe PINGR's design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems.
  
  
  
  Eight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure.
  
  
  
  A final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were 'Consistency and standards' (13 usability issues; 28% of the total) and 'Match between system and real world' (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data identification from a figure (both population-level and patient-level); (3) Disagreement with a system recommendation.
  
  
  
  By contextualising our findings within the wider literature on health information system usability, we provide recommendations for the design of e-A&F system interfaces relating to their four key components, in addition to how they may be integrated within a system.
authors: Health eResearch Centre, Farr Institute of Health Informatics Research, Centre for Health Informatics, University of Manchester, Manchester, UK. Electronic address: benjamin.brown@manchester.ac.uk.
authors: Benjamin
authors: B
authors: Brown
authors: Health eResearch Centre, Farr Institute of Health Informatics Research, Centre for Health Informatics, University of Manchester, Manchester, UK.
authors: Panos
authors: P
authors: Balatsoukas
authors: Health eResearch Centre, Farr Institute of Health Informatics Research, Centre for Health Informatics, University of Manchester, Manchester, UK.
authors: Richard
authors: R
authors: Williams
authors: Health eResearch Centre, Farr Institute of Health Informatics Research, Centre for Health Informatics, University of Manchester, Manchester, UK.
authors: Matthew
authors: M
authors: Sperrin
authors: Health eResearch Centre, Farr Institute of Health Informatics Research, Centre for Health Informatics, University of Manchester, Manchester, UK.
authors: Iain
authors: I
authors: Buchan
mesh_terms: D054869:Clinical Audit; D003201:Computers; D005246:Feedback; D063005:Health Information Systems; D000066506:Heuristics; D006801:Humans; D011320:Primary Health Care; D058996:Quality Improvement; D011787:Quality of Health Care / Q000592:standards*; D012106:Research; D014584:User-Computer Interface
keywords: Clinical audit; Clinical decision support systems; Clinical governance; Clinical quality improvement; Clinical quality management; Hybrid usability inspection; Medical audit; Usability studies; User interface design