title: A behaviourally anchored rating scale for evaluating the use of the WHO surgical safety checklist: development and initial evaluation of the WHOBARS.
abstract: Realising the full potential of the WHO Surgical Safety Checklist (SSC) to reduce perioperative harm requires the constructive engagement of all operating room (OR) team members during its administration. To facilitate research on SSC implementation, a valid and reliable instrument is needed for measuring OR team behaviours during its administration. We developed a behaviourally anchored rating scale (BARS) for this purpose.
  
  
  
  We used a modified Delphi process, involving 16 subject matter experts, to compile a BARS with behavioural domains applicable to all three phases of the SSC. We evaluated the instrument in 80 adult OR cases and 30 simulated cases using two medical student raters and seven expert raters, respectively. Intraclass correlation coefficients were calculated to assess inter-rater reliability. Internal consistency and instrument discrimination were explored. Sample size estimates for potential study designs using the instrument were calculated.
  
  
  
  The Delphi process resulted in a BARS instrument (the WHOBARS) with five behavioural domains. Intraclass correlation coefficients calculated from the OR cases exceeded 0.80 for 80% of the instrument's domains across the SSC phases. The WHOBARS showed high internal consistency across the three phases of the SSC and ability to discriminate among surgical cases in both clinical and simulated settings. Fewer than 20 cases per group would be required to show a difference of 1 point between groups in studies of the SSC, where α=0.05 and β=0.8.
  
  
  
  We have developed a generic instrument for comprehensively rating the administration of the SSC and informing initiatives to realise its full potential. We have provided data supporting its capacity for discrimination, internal consistency and inter-rater reliability. Further psychometric evaluation is warranted.
authors.affiliation: Department of Anaesthesiology, University of Auckland, Auckland, New Zealand.
authors.forename: Daniel A
authors.initials: DA
authors.lastname: Devcich
authors.affiliation: Department of Anaesthesiology, University of Auckland, Auckland, New Zealand Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
authors.forename: Jennifer
authors.initials: J
authors.lastname: Weller
authors.affiliation: Department of Anaesthesiology, University of Auckland, Auckland, New Zealand Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
authors.forename: Simon J
authors.initials: SJ
authors.lastname: Mitchell
authors.affiliation: School of Medicine, University of Auckland, Auckland, New Zealand.
authors.forename: Scott
authors.initials: S
authors.lastname: McLaughlin
authors.affiliation: School of Medicine, University of Auckland, Auckland, New Zealand.
authors.forename: Lauren
authors.initials: L
authors.lastname: Barker
authors.affiliation: Harvard Medical School, Center for Medical Simulation, Boston, Massachusetts, United States.
authors.forename: Jenny W
authors.initials: JW
authors.lastname: Rudolph
authors.affiliation: Harvard Medical School, Center for Medical Simulation, Boston, Massachusetts, United States.
authors.forename: Daniel B
authors.initials: DB
authors.lastname: Raemer
authors.affiliation: Harvard Medical School, Brigham and Women's Hospital, Boston, Massachusetts, United States.
authors.forename: Martin
authors.initials: M
authors.lastname: Zammert
authors.affiliation: Department of Health Policy and Management, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, United States.
authors.forename: Sara J
authors.initials: SJ
authors.lastname: Singer
authors.affiliation: Department of Anaesthesiology, University of Auckland, Auckland, New Zealand Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
authors.forename: Jane
authors.initials: J
authors.lastname: Torrie
authors.affiliation: Department of Medicine, Christchurch School of Medicine and Health Sciences, University of Otago, Auckland, New Zealand.
authors.forename: Chris Ma
authors.initials: CM
authors.lastname: Frampton
authors.affiliation: Department of Anaesthesiology, University of Auckland, Auckland, New Zealand Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand.
authors.forename: Alan F
authors.initials: AF
authors.lastname: Merry
mesh_terms: D057189:Checklist / Q000592:standards*; D003142:Communication; D006801:Humans; D009873:Operating Rooms / Q000458:organization & administration* / Q000592:standards; D010348:Patient Care Team / Q000458:organization & administration* / Q000592:standards; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires / Q000592:standards*; D014944:World Health Organization
keywords: Checklists; Communication; Patient safety; Surgery; Teamwork