Should non-expert clinician examiners be used in objective structured assessment of communication skills among final year medical undergraduates?
BACKGROUND
Adoption of the objective structured clinical examination may be hindered by shortages of clinicians within a specialty. Clinicians from other specialties should be considered as alternative, non-expert examiners.


OBJECTIVE
We assessed the inter-rater agreement between expert and non-expert clinician examiners in an integrated objective structured clinical examination for final year medical undergraduates.


METHODS
Pairs of expert and non-expert clinician examiners used a rating checklist to assess students in 8 oral communication stations, representing commonly encountered scenarios from medicine, paediatrics, and surgery. These included breaking bad news, managing an angry relative, taking consent for lumbar puncture; and advising a mother on asthma and febrile fits, and an adult on medication use, lifestyle changes and post-suture care of a wound. 439 students participated in the OSCE (206 in 2005, 233 in 2006).


RESULTS
There was good to very good agreement (intraclass coefficient: 0.57-0.79) between expert and non-expert clinician examiners, with 5 out of 8 stations having intraclass coefficients > or =0.70. Variation between paired examiners within stations contributed the lowest variance to student scores.


CONCLUSIONS
These findings support the use of clinicians from other specialties, as 'non-expert' examiners, to assess communication skills, using a standardized checklist, thereby reducing the demand on clinicians' time.
D002983:Clinical Competence / Q000592:standards*; D003142:Communication; D004504:Education, Medical, Undergraduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D005180:Faculty, Medical / Q000592:standards / Q000600:supply & distribution; D006801:Humans; D015588:Observer Variation; D010865:Pilot Projects; D015203:Reproducibility of Results