Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination.
BACKGROUND
One of the methods used for standard setting is the borderline regression method (BRM). This study aims to assess the reliability of BRM when the pass-fail standard in an objective structured clinical examination (OSCE) was calculated by averaging the BRM standards obtained for each station separately.


METHODS
In nine stations of the OSCE with direct observation the examiners gave each student a checklist score and a global score. Using a linear regression model for each station, we calculated the checklist score cut-off on the regression equation for the global scale cut-off set at 2. The OSCE pass-fail standard was defined as the average of all station's standard. To determine the reliability, the root mean square error (RMSE) was calculated. The R (2) coefficient and the inter-grade discrimination were calculated to assess the quality of OSCE.


RESULTS
The mean total test score was 60.78. The OSCE pass-fail standard and its RMSE were 47.37 and 0.55, respectively. The R (2) coefficients ranged from 0.44 to 0.79. The inter-grade discrimination score varied greatly among stations.


CONCLUSIONS
The RMSE of the standard was very small indicating that BRM is a reliable method of setting standard for OSCE, which has the advantage of providing data for quality assurance.
Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
Sara Mortaz
SM
Hejri
Department of Emergency Medicine, Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
Mohammad
M
Jalili
Department of Medical Education and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands.
Arno M M
AM
Muijtjens
Department of Educational Development and Research, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands.
Cees P M
CP
Van Der Vleuten
Borderline regression method; objective structured clinical examination; reliability; standard setting