title: Group assessments of resident physicians improve reliability and decrease halo error.
abstract: BACKGROUND
  Individual faculty assessments of resident competency are complicated by inconsistent application of standards, lack of reliability, and the "halo" effect.
  
  
  OBJECTIVE
  We determined whether the addition of faculty group assessments of residents in an ambulatory clinic, compared with individual faculty-of-resident assessments alone, have better reliability and reduced halo effects.
  
  
  METHODS
  This prospective, longitudinal study was performed in the outpatient continuity clinics of a large internal medicine residency program.
  
  
  METHODS
  Faculty-on-resident and group faculty-on-resident assessment scores were used for comparison.
  
  
  RESULTS
  Overall mean scores were significantly higher for group than individual assessments (3.92 ± 0.51 vs. 3.83 ± 0.38, p = 0.0001). Overall inter-rater reliability increased when combining group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% CI = 0.828, 0.785-0.866 vs. 0.749, 0.686-0.804). Inter-item correlations were less for group (0.49) than individual (0.68) assessments.
  
  
  CONCLUSIONS
  This study demonstrates improved inter-rater reliability and reduced range restriction (halo effect) of resident assessment across multiple performance domains by adding the group assessment method to traditional individual faculty-on-resident assessment. This feasible model could help graduate medical education programs achieve more reliable and discriminating resident assessments.
authors.affiliation: Division of Primary Care Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, USA. thomas.matthew@mayo.edu
authors.forename: Matthew R
authors.initials: MR
authors.lastname: Thomas
authors.forename: Thomas J
authors.initials: TJ
authors.lastname: Beckman
authors.forename: Karen F
authors.initials: KF
authors.lastname: Mauck
authors.forename: Stephen S
authors.initials: SS
authors.lastname: Cha
authors.forename: Kris G
authors.initials: KG
authors.lastname: Thomas
mesh_terms: D000704:Analysis of Variance; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000379:methods* / Q000592:standards; D004521:Educational Measurement; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency / Q000592:standards*; D008137:Longitudinal Studies; D010379:Peer Group; D011446:Prospective Studies; D015203:Reproducibility of Results