title: Reliability of Verbal Handoff Assessment and Handoff Quality Before and After Implementation of a Resident Handoff Bundle.
abstract: 1) To develop validity evidence for the use of the Verbal Handoff Assessment Tool (VHAT) and examine the reliability of VHAT scores, and 2) to determine whether implementation of a resident handoff bundle (RHB) was associated with improved verbal patient handoffs among pediatric resident physicians.
  
  
  
  In a pre-post design, prospectively audio recorded verbal patient handoffs conducted at Boston Children's Hospital before and after implementation of the RHB were rated using the VHAT, which was developed for this study (primary outcome). Using generalizability theory, we evaluated the reliability of VHAT scores.
  
  
  
  Overall, VHAT scores increased after RHB implementation (mean 142 vs 191, possible score 0-500; P < .0001). When accounting for clustering according to resident physician, hospital unit, unit census, and patient complexity, implementation of the RHB was associated with a 63-point increase in VHAT score. Using generalizability theory, we determined that a resident's mean VHAT score on the basis of a handoff of 15 patients assessed by a single observer was sufficiently reliable for relative ranking decisions (ie, norm-based; generalizability coefficient, 0.81), whereas a VHAT score on the basis of a handoff of 21 patients would be sufficiently reliable for high-stakes, standard-based decisions (Phi, 0.80).
  
  
  
  Verbal handoffs improved after implementation of a RHB, although gains were variable across the 2 clinical units. The VHAT shows promise as an assessment tool for resident handoff skills. If used for competency or entrustment decisions, a resident's mean VHAT score should be on the basis of observation of verbal handoff of ≥21 patients.
authors.affiliation: Department of Medicine, Boston Children's Hospital, Boston, Mass; Department of Pediatric Hematology/Oncology, Dana-Farber/Boston Children's Cancer and Blood Disorders Center, Boston, Mass; Department of Pediatrics, Harvard Medical School, Boston, Mass. Electronic address: Angela_Feraco@dfci.harvard.edu.
authors.forename: Angela M
authors.initials: AM
authors.lastname: Feraco
authors.affiliation: Department of Medicine, Boston Children's Hospital, Boston, Mass; Department of Pediatrics, Harvard Medical School, Boston, Mass.
authors.forename: Amy J
authors.initials: AJ
authors.lastname: Starmer
authors.affiliation: Department of Medicine, Boston Children's Hospital, Boston, Mass; Department of Pediatrics, Harvard Medical School, Boston, Mass.
authors.forename: Theodore C
authors.initials: TC
authors.lastname: Sectish
authors.affiliation: St. Christopher's Hospital for Children, Philadelphia, Penn; Department of Pediatrics, Drexel University College of Medicine, Philadelphia, Penn.
authors.forename: Nancy D
authors.initials: ND
authors.lastname: Spector
authors.affiliation: UCSF Benioff Children's Hospital and Department of Pediatrics, University of California, San Francisco, Calif.
authors.forename: Daniel C
authors.initials: DC
authors.lastname: West
authors.affiliation: Department of Medicine, Boston Children's Hospital, Boston, Mass; Department of Pediatrics, Harvard Medical School, Boston, Mass; Sleep and Patient Safety Program, Brigham and Women's Hospital, Boston, Mass.
authors.forename: Christopher P
authors.initials: CP
authors.lastname: Landrigan
mesh_terms: D001900:Boston; D003142:Communication; D065187:Controlled Before-After Studies; D004650:Employee Performance Appraisal; D006776:Hospitals, Pediatric; D006801:Humans; D007396:Internship and Residency; D062209:Patient Handoff / Q000458:organization & administration / Q000592:standards*; D010372:Pediatrics / Q000193:education*; D015203:Reproducibility of Results
keywords: graduate medical education; handoff communication; patient handoffs; pediatric residents; workplace-based assessment