title: Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors.
abstract: BACKGROUND
  Multi-source evaluation has demonstrated value for trainees, but is not generally provided to residency or fellowship program directors (PDs).
  
  
  OBJECTIVE
  To develop, implement, and evaluate a PD multi-source evaluation process.
  
  
  METHODS
  Tools were developed for PD evaluation by trainees, department chairs, and graduate medical education (GME) leadership. Evaluation questions were based on PD responsibilities, including Accreditation Council for Graduate Medical Education (ACGME) requirements. A follow-up survey assessed the process.
  
  
  RESULTS
  Evaluation completion rates were as follows: trainees in academic year 2012-2013, 53% (958 of 1824), and in academic year 2013-2014, 42% (800 of 1898); GME directors in 2013-2014, 100% (95 of 95); and chairs/chiefs in 2013-2014, 92% (109 of 118). Results of a follow-up survey of PDs (66%, 59 of 90) and chairs (74%, 48 of 65) supports the evaluations' value, with 45% of responding PDs (25 of 56) and 50% of responding chairs (21 of 42) characterizing them as "extremely" or "quite" useful. Most indicated this was the first written evaluation they had received (PDs 78%, 46 of 59) or provided (chairs 69%, 33 of 48) regarding the PD role. More than 60% of PD (30 of 49) and chair respondents (24 of 40) indicated trainee feedback was "extremely" or "quite" useful, and nearly 50% of PDs (29 of 59) and 21% of chairs (10 of 48) planned changes based on the results. Trainee response rates improved in 2014-2015 (52%, 971 of 1872) and 2015-2016 (69%, 1276 of 1837).
  
  
  CONCLUSIONS
  In our institution, multi-source evaluation of PDs was sustained over 4 years with acceptable and improving evaluation completion rates. The process and assessment tools are potentially transferrable to other institutions.
authors: Mary Ellen J
authors: ME
authors: Goldhamer
authors: Keith
authors: K
authors: Baker
authors: Amy P
authors: AP
authors: Cohen
authors: Debra F
authors: DF
authors: Weinstein
mesh_terms: D000068:Accreditation; D004503:Education, Medical, Graduate / Q000458:organization & administration*; D005069:Evaluation Studies as Topic; D005257:Fellowships and Scholarships; D006801:Humans; D007396:Internship and Residency / Q000458:organization & administration*; D007857:Leadership; D008486:Physician Executives / Q000458:organization & administration*; D011795:Surveys and Questionnaires