The impact of the inclusion of simulated patient ratings on the reliability of  OSCE assessments under the borderline regression method.
BACKGROUND
In Objective Structured Clinical Examinations (OSCEs), the use of simulated patients (SPs) at many stations is a key aspect of the assessment. Often the SPs are asked to provide formal feedback (ratings) of their experience with the students under examination.


OBJECTIVE
This study analyses whether and how exactly SP data can be best used to enhance the robustness of the formal standard setting process.


METHODS
A retrospective statistical investigation into the relationship between SP ratings and those provided by the clinical assessors (criterion-based checklist scores and overall grades for each station) is presented. In addition, the paper also includes a study into the impact of the inclusion of the SP ratings in the formal standard setting process for OSCEs, particularly when pass marks are calculated using the borderline regression method.


RESULTS
The general results of the analysis, including discussion of two distinct methods for the combining of the SP ratings and assessor judgements, are presented, and demonstrate that the inclusion of this additional data can have important effects on individual student results.


CONCLUSIONS
It is possible for the overall quality of the OSCE assessment process to be improved, with increased reliability, by combining assessor checklist scores and SP ratings.
Homer
Matthew
M
Medical Education Unit, School of Medicine, University of Leeds, Leeds, UK. m.s.homer@leeds.ac.uk
Pell
Godfrey
G
D057189:Checklist; D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D016544:Patient Simulation; D012189:Retrospective Studies