title: Assessing professionalism in the context of an objective structured clinical examination: an in-depth study of the rating process.
abstract: BACKGROUND
  Professionalism is fundamental to the practice of medicine. Objective structured clinical examinations (OSCEs) have been proposed as appropriate for assessing some aspects of professionalism. This study investigated how raters assign professionalism ratings to medical students' performances in OSCE encounters.
  
  
  METHODS
  Three standardised patients, 3 doctor preceptors, and 3 lay people viewed and rated 20 videotaped encounters between 3rd-year medical students and standardised patients. Raters recorded their thoughts while rating. Qualitative and quantitative analyses were conducted. Comments about observable behaviours were coded, and relative frequencies were computed. Correlations between counts of categorised comments and overall professionalism ratings were also computed.
  
  
  RESULTS
  Raters varied in which behaviours they attended to, and how behaviours were evaluated. This was true within and between rater type. Raters also differed in the behaviours they consider when providing global evaluations of professionalism.
  
  
  CONCLUSIONS
  This study highlights the complexity of the processes involved in assigning ratings to doctor-patient encounters. Greater emphasis on behavioural definitions of specific behaviours may not be a sufficient solution, as raters appear to vary in both attention to and evaluation of behaviours. Reliance on global ratings is also problematic, especially if relatively few raters are used, for similar reasons. We propose a model highlighting the multiple points where raters viewing the same encounter may diverge, resulting in different ratings of the same performance. Progress in assessment of professionalism will require further dialogue about what constitutes professional behaviour in the medical encounter, with input from multiple constituencies and multiple representatives within each constituency.
authors: Meyers Primary Care Institute, University of Massachusetts Medical School, MA, USA. Kathleen.mazor@umassmed.edu
authors: Kathleen M
authors: KM
authors: Mazor
authors: Mary L
authors: ML
authors: Zanetti
authors: Eric J
authors: EJ
authors: Alper
authors: David
authors: D
authors: Hatem
authors: Susan V
authors: SV
authors: Barrett
authors: Vanessa
authors: V
authors: Meterko
authors: Wendy
authors: W
authors: Gammon
authors: Michele P
authors: MP
authors: Pugnaire
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003142:Communication; D004504:Education, Medical, Undergraduate; D006801:Humans; D008404:Massachusetts; D010817:Physician-Patient Relations; D013337:Students, Medical