Calibration of communication skills items in OSCE checklists according to the MAAS-Global.
BACKGROUND
Communication skills (CS) are commonly assessed using 'communication items' in Objective Structured Clinical Examination (OSCE) station checklists. Our aim is to calibrate the communication component of OSCE station checklists according to the MAAS-Global which is a valid and reliable standard to assess CS in undergraduate medical education.


METHODS
Three raters independently compared 280 checklists from 4 disciplines contributing to the undergraduate year 4 OSCE against the 17 items of the MAAS-Global standard. G-theory was used to analyze the reliability of this calibration procedure.


RESULTS
G-Kappa was 0.8. For two raters G-Kappa is 0.72 and it fell to 0.57 for one rater. 46% of the checklist items corresponded to section three of the MAAS-Global (i.e. medical content of the consultation), whilst 12% corresponded to section two (i.e. general CS), and 8.2% to section one (i.e. CS for each separate phase of the consultation). 34% of the items were not considered to be CS.


CONCLUSIONS
A G-Kappa of 0.8 confirms a reliable and valid procedure for calibrating OSCE CS checklist items using the MAAS-Global. We strongly suggest that such a procedure is more widely employed to arrive at a stable (valid and reliable) judgment of the communication component in existing checklists for medical students' communication behaviours.


CONCLUSIONS
It is possible to measure the 'true' caliber of CS in OSCE stations. Students' results are thereby comparable between and across stations, students and institutions. A reliable calibration procedure requires only two raters.
Faculty of Medicine and Health Sciences of the Universitas Muhammadiyah Yogyakarta, Indonesia.
Winny
W
Setyonugroho
School of Medicine, College or Medicine, Nursing & Health Sciences, National University of Ireland Galway, Ireland. Electronic address: thomas.kropmans@nuigalway.ie.
Thomas
T
Kropmans
School of Medicine, College or Medicine, Nursing & Health Sciences, National University of Ireland Galway, Ireland.
Kieran M
KM
Kennedy
School of Medicine, College or Medicine, Nursing & Health Sciences, National University of Ireland Galway, Ireland.
Brian
B
Stewart
Skills laboratory, Faculty of Health, Medicine and Life Sciences, Maastricht University, the Netherlands.
Jan
J
van Dalen
D057189:Checklist; D003142:Communication; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods / Q000592:standards*; D006801:Humans; D011361:Professional Competence / Q000592:standards*; D015203:Reproducibility of Results; D013337:Students, Medical; D011795:Surveys and Questionnaires
Communication skills; MAAS-Global; OSCE; Objective structured clinical examination; Reliability; Validity