title: Use of the mini-clinical evaluation exercise to rate examinee performance on a multiple-station clinical skills examination: a validity study.
abstract: BACKGROUND
  Multivariate generalizability analysis was used to investigate the performance of a commonly used clinical evaluation tool.
  
  
  METHODS
  Practicing physicians were trained to use the mini-Clinical Skills Examination (CEX) rating form to rate performances from the United States Medical Licensing Examination Step 2 Clinical Skills examination.
  
  
  RESULTS
  Differences in rater stringency made the greatest contribution to measurement error; more raters rating each examinee, even on fewer occasions, could enhance score stability. Substantial correlated error across the competencies suggests that decisions about one scale unduly influence those on others.
  
  
  CONCLUSIONS
  Given the appearance of a halo effect across competencies, score interpretations that assume assessment of distinct dimensions of clinical performance should be made with caution. If the intention is to produce a single composite score by combining results across competencies, the presence of these effects may be less critical.
authors.affiliation: National Board of Medical Examiners, 3750 Market Street, Philadelphia, Pennsylvania 19104, USA. mmargolis@nbme.org
authors.forename: Melissa J
authors.initials: MJ
authors.lastname: Margolis
authors.forename: Brian E
authors.initials: BE
authors.lastname: Clauser
authors.forename: Monica M
authors.initials: MM
authors.lastname: Cuddy
authors.forename: Andrea
authors.initials: A
authors.lastname: Ciccone
authors.forename: Janet
authors.initials: J
authors.lastname: Mee
authors.forename: Polina
authors.initials: P
authors.lastname: Harik
authors.forename: Richard E
authors.initials: RE
authors.lastname: Hawkins
mesh_terms: D000704:Analysis of Variance; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007407:Interviews as Topic; D010808:Physical Examination / Q000379:methods*; D012984:Software