title: Order effects in high stakes undergraduate examinations: an analysis of 5 years of administrative data in one UK medical school.
abstract: To investigate the association between student performance in undergraduate objective structured clinical examinations (OSCEs) and the examination schedule to which they were assigned to undertake these examinations.
  
  
  
  Analysis of routinely collected data.
  
  
  
  One UK medical school.
  
  
  
  2331 OSCEs of 3 different types (obstetrics OSCE, paediatrics OSCE and simulated clinical encounter examination OSCE) between 2009 and 2013. Students were not quarantined between examinations.
  
  
  
  (1) Pass rates by day examination started, (2) pass rates by day station undertaken and (3) mean scores by day examination started.
  
  
  
  We found no evidence that pass rates differed according to the day on which the examination was started by a candidate in any of the examinations considered (p>0.1 for all). There was evidence (p=0.013) that students were more likely to pass individual stations on the second day of the paediatrics OSCE (OR 1.27, 95% CI 1.05 to 1.54). In the cases of the simulated clinical encounter examination and the obstetrics and gynaecology OSCEs, there was no (p=0.42) or very weak evidence (p=0.099), respectively, of any such variation in the probability of passing individual stations according to the day they were attempted. There was no evidence that mean scores varied by day apart from the paediatric OSCE, where slightly higher scores were achieved on the second day of the examination.
  
  
  
  There is little evidence that different examination schedules have a consistent effect on pass rates or mean scores: students starting the examinations later were not consistently more or less likely to pass or score more highly than those starting earlier. The practice of quarantining students to prevent communication with (and subsequent unfair advantage for) subsequent examination cohorts is unlikely to be required.
authors.affiliation: Primary Care Unit, Cambridge Centre for Health Services Research, Cambridge, UK.
authors.forename: Jenni
authors.identifier: 0000-0002-0037-274X
authors.initials: J
authors.lastname: Burt
authors.affiliation: Primary Care Unit, Cambridge Centre for Health Services Research, Cambridge, UK University of Exeter Medical School, Exeter, UK.
authors.forename: Gary
authors.identifier: 0000-0003-2231-5161
authors.initials: G
authors.lastname: Abel
authors.affiliation: Primary Care Unit, Cambridge Centre for Health Services Research, Cambridge, UK.
authors.forename: Matt
authors.initials: M
authors.lastname: Barclay
authors.affiliation: Primary Care Unit, Institute of Public Health, Cambridge, UK.
authors.forename: Robert
authors.initials: R
authors.lastname: Evans
authors.affiliation: Primary Care Unit, Institute of Public Health, Cambridge, UK.
authors.forename: John
authors.initials: J
authors.lastname: Benson
authors.affiliation: Wellcome Trust-MRC Institute of Metabolic Science, NIHR Cambridge Biomedical Research Centre & University of Cambridge School of Clinical Medicine, Cambridge, UK.
authors.forename: Mark
authors.initials: M
authors.lastname: Gurnell
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D016001:Confidence Intervals; D004504:Education, Medical, Undergraduate / Q000592:standards / Q000706:statistics & numerical data*; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008297:Male; D008487:Medical History Taking / Q000592:standards; D009774:Obstetrics / Q000193:education / Q000592:standards; D010372:Pediatrics / Q000193:education / Q000592:standards; D010808:Physical Examination / Q000592:standards; D015203:Reproducibility of Results; D012577:Schools, Medical; D013337:Students, Medical; D006113:United Kingdom
keywords: Clinical Competence; Educational Measurement; MEDICAL EDUCATION & TRAINING