title: Does Making the Numerical Values of Verbal Anchors on a Rating Scale Available to Examiners Inflate Scores on a Long Case Examination?
abstract: OBJECTIVE
  Rating scales are frequently used for scoring assessments in medical education. The effect of changing the structural elements of a rating scale on students' examination scores has received little attention in the medical education literature. This study assessed the impact of making the numerical values of verbal anchors on a rating scale available to examiners in a long case examination (LCE).
  
  
  METHODS
  During the 2011-2012 academic year, the numerical values of verbal anchors on a rating scale for an internal medicine clerkship LCE were made available to faculty examiners. Historically, and specifically in the control year of 2010-2011, examiners only saw the scale's verbal anchors and were blinded to the associated numerical values. To assess the impact of this change, the authors compared students' LCE scores between the two cohort years. To assess for differences between the two cohorts, they compared students' scores on other clerkship assessments, which remained the same between the two cohorts.
  
  
  RESULTS
  From 2010-2011 (n = 226) to 2011-2012 (n = 218), the median LCE score increased significantly from 82.11% to 85.02% (P < .01). Students' performance on the other clerkship assessments was similar between cohorts.
  
  
  CONCLUSIONS
  Providing examiners with the numerical values of verbal anchors on a rating scale, in addition to the verbal anchors themselves, led to a significant increase in students' scores on an internal medicine clerkship LCE. When constructing or changing rating scales, educators must consider the potential impact of the rating scale structure on students' scores.
authors.affiliation: L.A. Devine is assistant professor, Department of Medicine, University of Toronto, and a member of the Ho Ping Kong Centre for Excellence in Education and Practice, University Health Network, Toronto, Ontario, Canada. L. Stroud is assistant professor, Department of Medicine, University of Toronto, and education researcher, Wilson Centre for Research in Education, University Health Network, Toronto, Ontario, Canada. R. Gupta is assistant professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada. E. Lorens is education research and evaluation officer, Department of Medicine, University of Toronto, Toronto, Ontario, Canada. S. Robertson is undergraduate medical education coordinator, Department of Medicine, University of Toronto, Toronto, Ontario, Canada. D. Panisko is professor, Department of Medicine, University of Toronto, and a member of the Ho Ping Kong Centre for Excellence in Education and Practice, University Health Network, Toronto, Ontario, Canada.
authors.forename: Luke A
authors.initials: LA
authors.lastname: Devine
authors.forename: Lynfa
authors.initials: L
authors.lastname: Stroud
authors.forename: Rajesh
authors.initials: R
authors.lastname: Gupta
authors.forename: Edmund
authors.initials: E
authors.lastname: Lorens
authors.forename: Sumitra
authors.initials: S
authors.lastname: Robertson
authors.forename: Daniel
authors.initials: D
authors.lastname: Panisko
mesh_terms: D002170:Canada; D002982:Clinical Clerkship; D004521:Educational Measurement / Q000706:statistics & numerical data*; D005180:Faculty, Medical; D006801:Humans; D007388:Internal Medicine / Q000193:education; D013337:Students, Medical