title: Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students.
abstract: BACKGROUND
  Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty.
  
  
  OBJECTIVE
  We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees).
  
  
  METHODS
  Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year.
  
  
  METHODS
  The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives.
  
  
  METHODS
  Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients.
  
  
  RESULTS
  The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21).
  
  
  CONCLUSIONS
  On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.
authors.affiliation: F. Edward Hébert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, MD, USA, paul.hemmer@usuhs.edu.
authors.forename: Paul A
authors.initials: PA
authors.lastname: Hemmer
authors.forename: Gregory A
authors.initials: GA
authors.lastname: Dadekian
authors.forename: Christopher
authors.initials: C
authors.lastname: Terndrup
authors.forename: Louis N
authors.initials: LN
authors.lastname: Pangaro
authors.forename: Allison B
authors.initials: AB
authors.lastname: Weisbrod
authors.forename: Mark D
authors.initials: MD
authors.lastname: Corriere
authors.forename: Rechell
authors.initials: R
authors.lastname: Rodriguez
authors.forename: Patricia
authors.initials: P
authors.lastname: Short
authors.forename: William F
authors.initials: WF
authors.lastname: Kelly
mesh_terms: D000328:Adult; D002982:Clinical Clerkship; D002983:Clinical Competence; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods*; D005180:Faculty, Medical; D005260:Female; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D008297:Male; D012189:Retrospective Studies