title: Do preceptors with more rating experience provide more reliable assessments of medical student performance?
abstract: BACKGROUND
  Although the existing psychometric literature provides guidance on the best method for acquiring a reliable clinical evaluation form (CEF)-based score, it also shows that a single CEF rating has very low reliability.
  
  
  OBJECTIVE
  This study examines whether experience with rating students might act as a form of rater training and hence improve the quality of CEF ratings.
  
  
  METHODS
  Preceptors were divided into two groups based on rater experience. The univariate and multivariate G study designs used were simple rater (r)-nested-within-person (p) [r : p and r(○) : p(•)] models, and in the univariate analysis was applied separately to CEFs completed by high and low experienced raters.
  
  
  RESULTS
  The high experienced rater group yielded a substantially higher observed reliability in both the univariate and multivariate analyses.
  
  
  CONCLUSIONS
  These results support the hypothesis that high experienced raters produce more reliable ratings of student performance and suggest methods for improving CEF ratings.
authors.affiliation: General Internal Medicine, University of Iowa Carver College of Medicine, Iowa City, Iowa 52245, USA. kristiferguson@uiowa.edu
authors.forename: Kristi J
authors.initials: KJ
authors.lastname: Ferguson
authors.forename: Clarence D
authors.initials: CD
authors.lastname: Kreiter
authors.forename: Rick D
authors.initials: RD
authors.lastname: Axelson
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000592:standards*; D005180:Faculty, Medical; D006801:Humans; D007484:Iowa; D011231:Preceptorship / Q000592:standards*; D013337:Students, Medical