The method used to set the pass mark in an objective structured clinical examination defines the performance of candidates for certification as rheumatologists.
BACKGROUND
The Mexican Accreditation Council for Rheumatology certifies trainees (TR) on an annual basis using both a multiple-choice question (MCQ) test and an objective structured clinical examination (OSCE). For 2013 and 2014, the OSCE pass mark (PM) was set by criterion referencing as ≥6 (CPM), whereas overall rating of borderline performance method (BPM) was added for 2015 and 2016 accreditations. We compared OSCE TR performance according to CPM and BPM, and examined whether correlations between MCQ and OSCE were affected by PM.


METHODS
Forty-three (2015) and 37 (2016) candidates underwent both tests. Altogether, OSCE were integrated by 15 validated stations; one evaluator per station scored TR performance according to a station-tailored check-list and a Likert scale (fail, borderline, above range) of overall performance. A composite OSCE score was derived for each candidate. Appropriate statistics were used.


RESULTS
Mean (±standard derivation [SD]) MCQ test scores were 6.6±0.6 (2015) and 6.4±0.6 (2016) with 5 candidates receiving a failing score each year. Mean (±SD) OSCE scores were 7.4±0.6 (2015) and 7.3±0.6 (2016); no candidate received a failing CPM score in either 2015 or 2016 OSCE, although 21 (49%) and 19 (51%) TR, respectively, received a failing BPM score (calculated as 7.3 and 7.4, respectively). Stations for BPM ranged from 4.5 to 9.5; overall, candidates showed better performance in CPM. In all, MCQ correlated with composite OSCE, r=0.67 (2015) and r=0.53 (2016); P≤.001. Trainees with a passing BPM score in OSCE had higher MCQ scores than those with a failing score.


CONCLUSIONS
Overall, OSCE-PM selection impacted candidates' performance but had a limited affect on correlation between clinical and practical examinations.
Consejo Mexicano de Reumatología, Ciudad de México, México. Electronic address: virtichu@gmail.com.
Virginia
V
Pascual-Ramos
Consejo Mexicano de Reumatología, Ciudad de México, México.
Ana
A
Guilaisne Bernard-Medina
Consejo Mexicano de Reumatología, Ciudad de México, México.
Diana Elsa
DE
Flores-Alvarado
Consejo Mexicano de Reumatología, Ciudad de México, México.
Margarita
M
Portela-Hernández
Consejo Mexicano de Reumatología, Ciudad de México, México.
María Del Rocío
MDR
Maldonado-Velázquez
Consejo Mexicano de Reumatología, Ciudad de México, México.
Luis Javier
LJ
Jara-Quezada
Consejo Mexicano de Reumatología, Ciudad de México, México.
Luis Manuel
LM
Amezcua-Guerra
Consejo Mexicano de Reumatología, Ciudad de México, México.
Nadina E
NE
Rubio-Judith López-Zepeda
Consejo Mexicano de Reumatología, Ciudad de México, México.
Everardo
E
Álvarez-Hernandez
Consejo Mexicano de Reumatología, Ciudad de México, México.
Miguel Ángel
MÁ
Saavedra
Consejo Mexicano de Reumatología, Ciudad de México, México.
César Alejandro
CA
Arce-Salinas
Consejo Mexicano de Reumatología, Ciudad de México, México.
D000073978:Academic Performance / Q000592:standards*; D002568:Certification / Q000592:standards*; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods / Q000592:standards*; D006801:Humans; D008800:Mexico; D000072140:Rheumatologists / Q000193:education*; D012219:Rheumatology / Q000193:education
Accreditation; Certificación; Competencias basadas en educación; Education-based competence