Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees - a Canadian multi-centred study.
BACKGROUND
The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.


METHODS
EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE. Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen's Simulation Assessment Tool (QSAT). Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario. A fully crossed generalizability study was also conducted for each examination centre.


RESULTS
Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.


CONCLUSIONS
This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees.
Department of Emergency Medicine, Queen's University.
J Damon
JD
Dagnone
Department of Emergency Medicine, Queen's University.
Andrew K
AK
Hall
Faculty of Education, Queen's University.
Stefanie
S
Sebok-Syer
Faculty of Education, Queen's University.
Don
D
Klinger
Department of Emergency Medicine, Western University.
Karen
K
Woolfrey
Department of Emergency Medicine, Queen's University; Department of Public Health Sciences, Queen's University.
Colleen
C
Davison
Department of Emergency Medicine, Dalhousie University.
John
J
Ross
Department of Emergency Medicine University of Calgary.
Gordon
G
McNeil
Department of Emergency Medicine, University of Ottawa.
Sean
S
Moore