title: How faculty members experience workplace-based assessment rater training: a qualitative study.
abstract: BACKGROUND
  Direct observation of clinical skills is a common approach in workplace-based assessment (WBA). Despite widespread use of the mini-clinical evaluation exercise (mini-CEX), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members' reactions to rater training.
  
  
  OBJECTIVE
  This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches - performance dimension training (PDT) and a modified approach to frame of reference training (FoRT) - to elucidate how such faculty development can be optimally designed.
  
  
  METHODS
  In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out-patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data.
  
  
  RESULTS
  Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one's approach to assessment, and (iv) concerns about institutional culture and buy-in.
  
  
  CONCLUSIONS
  Rater training using PDT and a modified approach to FoRT can provide faculty staff with assessment skills that are congruent with principles of criterion-referenced assessment and entrustment, and foundational principles of competency-based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.
authors.affiliation: Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
authors.forename: Jennifer R
authors.initials: JR
authors.lastname: Kogan
authors.affiliation: Milestones Development and Evaluation, Accreditation Council of Graduate Medical Education, Chicago, Illinois, USA.
authors.forename: Lisa N
authors.initials: LN
authors.lastname: Conforti
authors.affiliation: Evaluation Research and Development, American Board of Internal Medicine, Philadelphia, Pennsylvania, USA.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Bernabeo
authors.affiliation: Academic and Clinical Affairs, Commonwealth Medical College, Scranton, Pennsylvania, USA.
authors.forename: William
authors.initials: W
authors.lastname: Iobst
authors.affiliation: Milestones Development and Evaluation, Accreditation Council of Graduate Medical Education, Chicago, Illinois, USA.
authors.forename: Eric
authors.initials: E
authors.lastname: Holmboe
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D003162:Competency-Based Education / Q000592:standards; D004503:Education, Medical, Graduate / Q000379:methods; D004521:Educational Measurement / Q000379:methods / Q000592:standards; D005180:Faculty, Medical / Q000592:standards*; D005246:Feedback; D005260:Female; D066296:Grounded Theory; D006801:Humans; D007388:Internal Medicine / Q000193:education* / Q000592:standards; D008297:Male; D008875:Middle Aged; D036301:Qualitative Research; D014481:United States; D017132:Workplace