title: Feasibility and reliability of a multisource feedback tool for emergency medicine residents.
abstract: BACKGROUND
  While the Accreditation Council for Graduate Medical Education recommends multisource feedback (MSF) of resident performance, there is no uniformly accepted MSF tool for emergency medicine (EM) trainees, and the process of obtaining MSF in EM residencies is untested.
  
  
  OBJECTIVE
  To determine the feasibility of an MSF program and evaluate the intraclass and interclass correlation of a previously reported resident professionalism evaluation, the Humanism Scale (HS).
  
  
  METHODS
  To assess 10 third-year EM residents, we distributed an anonymous 9-item modified HS (EM-HS) to emergency department nursing staff, faculty physicians, and patients. The evaluators rated resident performance on a 1 to 9 scale (needs improvement to outstanding). Residents were asked to complete a self-evaluation of performance, using the same scale.
  
  
  METHODS
  Generalizability coefficients (Eρ(2)) were used to assess the reliability within evaluator classes. The mean score for each of the 9 questions provided by each evaluator class was calculated for each resident. Correlation coefficients were used to evaluate correlation between rater classes for each question on the EM-HS. Eρ(2) and correlation values greater than 0.70 were deemed acceptable.
  
  
  RESULTS
  EM-HSs were obtained from 44 nurses and 12 faculty physicians. The residents had an average of 13 evaluations by emergency department patients. Reliability within faculty and nurses was acceptable, with Eρ(2) of 0.79 and 0.83, respectively. Interclass reliability was good between faculty and nurses.
  
  
  CONCLUSIONS
  An MSF program for EM residents is feasible. Intraclass reliability was acceptable for faculty and nurses. However, reliable feedback from patients requires a larger number of patient evaluations.
authors.forename: Gregory
authors.initials: G
authors.lastname: Garra
authors.forename: Andrew
authors.initials: A
authors.lastname: Wackett
authors.forename: Henry
authors.initials: H
authors.lastname: Thode