title: The effect on reliability and sensitivity to level of training of combining analytic and holistic rating scales for assessing communication skills in an internal medicine resident OSCE.
abstract: OBJECTIVE
  Although previous research has compared checklists to rating scales for assessing communication, the purpose of this study was to compare the effect on reliability and sensitivity to level of training of an analytic, a holistic, and a combined analytic-holistic rating scale in assessing communication skills.
  
  
  METHODS
  The University of Alberta Internal Medicine Residency runs OSCEs for postgraduate year (PGY) 1 and 2 residents and another for PGY-4 residents. Communication stations were scored with an analytic scale (empathy, non-verbal skills, verbal skills, and coherence subscales) and a holistic scale. Authors analyzed reliability of individual and combined scales using generalizability theory and evaluated each scale's sensitivity to level of training.
  
  
  RESULTS
  For analytic, holistic, and combined scales, 12, 12, and 11 stations respectively yielded a Phi of 0.8 for the PGY-1,2 cohort, and 16, 16, and 14 stations yielded a Phi of 0.8 for the PGY-4 cohort. PGY-4 residents scored higher on the combined scale, the analytic rating scale, and the non-verbal and coherence subscales.
  
  
  CONCLUSIONS
  A combined analytic-holistic rating scale increased score reliability and was sensitive to level of training.
  
  
  CONCLUSIONS
  Given increased validity evidence, OSCE developers should consider combining analytic and holistic scales when assessing communication skills.
authors.affiliation: Department of Medicine, University of Alberta, Edmonton, Canada. Electronic address: vdaniels@ualberta.ca.
authors.forename: Vijay John
authors.initials: VJ
authors.lastname: Daniels
authors.affiliation: School of Dentistry, University of Alberta, Edmonton, Canada.
authors.forename: Dwight
authors.initials: D
authors.lastname: Harley
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003142:Communication; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency; D011237:Predictive Value of Tests; D011594:Psychometrics / Q000379:methods*; D015203:Reproducibility of Results
keywords: Assessment; Communication skills; Internal medicine; OSCE; Postgraduate medical education; Rating scales; Standardized patients