title: Introducing a model for optimal design of sequential objective structured clinical examinations.
abstract: In a sequential OSCE which has been suggested to reduce testing costs, candidates take a short screening test and who fail the test, are asked to take the full OSCE. In order to introduce an effective and accurate sequential design, we developed a model for designing and evaluating screening OSCEs. Based on two datasets from a 10-station pre-internship OSCE and considering three factors, namely, the number of stations, the criteria for selecting the stations, and the cut-off score, several hypothetical tests were proposed. To investigate their accuracy, the positive predictive value (PPV), the pass rate, and the negative predictive value (NPV) were calculated. Also, a "desirable" composite outcome was defined as PPV = 100 %, pass rate ≥50 %, and NPV ≥25 %. Univariate and multiple logistic regression analyses were conducted to estimate the effects of independent factors on the occurrence of the desirable outcome. In half of the screening tests no false positive result was detected. Most of the screening OSCEs had acceptable levels of pass rate and NPV. Considering the desirable composite outcome 20 screening OSCEs could have successfully predicted the results of the corresponding full OSCE. The multiple regression analysis indicated significant contributions for the selection criteria (p values = 0.019) and the cut-off score (p values = 0.017). In order to have efficient screening OSCEs with the lowest probability of the error rate, careful selection of stations with high values of discrimination or item total correlation, and use of a relatively stringent cut-off score should be considered.
authors.affiliation: Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
authors.forename: Sara
authors.initials: S
authors.lastname: Mortaz Hejri
authors.affiliation: Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran.
authors.forename: Kamran
authors.initials: K
authors.lastname: Yazdani
authors.affiliation: Department of Emergency Medicine, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
authors.forename: Ali
authors.initials: A
authors.lastname: Labaf
authors.affiliation: Foundation for Advancement of International Medical Education and Research, Philadelphia, PA, USA.
authors.forename: John J
authors.initials: JJ
authors.lastname: Norcini
authors.affiliation: Department of Emergency Medicine, Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, No. 57, Hojatdust St., Keshavarz Blvd, Tehran, Iran. mjalili@tums.ac.ir.
authors.forename: Mohammad
authors.initials: M
authors.lastname: Jalili
mesh_terms: D004504:Education, Medical, Undergraduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007492:Iran; D015233:Models, Statistical; D015203:Reproducibility of Results; D012189:Retrospective Studies
keywords: Objective structured clinical examination; Screening test; Sequential design