title: Do OSCE progress test scores predict performance in a national high-stakes examination?
abstract: BACKGROUND
  Progress tests, in which learners are repeatedly assessed on equivalent content at different times in their training and provided with feedback, would seem to lend themselves well to a competency-based framework, which requires more frequent formative assessments. The objective structured clinical examination (OSCE) progress test is a relatively new form of assessment that is used to assess the progression of clinical skills. The purpose of this study was to establish further evidence for the use of an OSCE progress test by demonstrating an association between scores from this assessment method and those from a national high-stakes examination.
  
  
  METHODS
  The results of 8 years' of data from an Internal Medicine Residency OSCE (IM-OSCE) progress test were compared with scores on the Royal College of Physicians and Surgeons of Canada Comprehensive Objective Examination in Internal Medicine (RCPSC IM examination), which is comprised of both a written and performance-based component (n = 180). Correlations between scores in the two examinations were calculated. Logistic regression analyses were performed comparing IM-OSCE progress test scores with an 'elevated risk of failure' on either component of the RCPSC IM examination.
  
  
  RESULTS
  Correlations between scores from the IM-OSCE (for PGY-1 residents to PGY-4 residents) and those from the RCPSC IM examination ranged from 0.316 (p = 0.001) to 0.554 (<.001) for the performance-based component and 0.305 (p = 0.002) to 0.516 (p < 0.001) for the written component. Logistic regression models demonstrated that PGY-2 and PGY-4 scores from the IM-OSCE were predictive of an 'elevated risk of failure' on both components of the RCPSC IM examination.
  
  
  CONCLUSIONS
  This study provides further evidence for the use of OSCE progress testing by demonstrating a correlation between scores from an OSCE progress test and a national high-stakes examination. Furthermore, there is evidence that OSCE progress test scores are predictive of future performance on a national high-stakes examination.
authors.affiliation: Department of Medicine, The Ottawa Hospital, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Ontario, Canada.
authors.forename: Debra
authors.initials: D
authors.lastname: Pugh
authors.affiliation: Department of Pediatrics, McGill University, Montreal, Quebec, Canada.|Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada.
authors.forename: Farhan
authors.initials: F
authors.lastname: Bhanji
authors.affiliation: Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada.
authors.forename: Gary
authors.initials: G
authors.lastname: Cole
authors.affiliation: Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Dupre
authors.affiliation: Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.
authors.forename: Rose
authors.initials: R
authors.lastname: Hatala
authors.affiliation: Department of Medicine, The Ottawa Hospital, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Ontario, Canada.|Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada.
authors.forename: Susan
authors.initials: S
authors.lastname: Humphrey-Murto
authors.affiliation: Department of Medicine, The Ottawa Hospital, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Ontario, Canada.|Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada.|Medical Council of Canada, Ottawa, Ontario, Canada.
authors.forename: Claire
authors.initials: C
authors.lastname: Touchie
authors.affiliation: Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada.
authors.forename: Timothy J
authors.initials: TJ
authors.lastname: Wood
mesh_terms: D002170:Canada; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D007388:Internal Medicine / Q000193:education; D007396:Internship and Residency / Q000592:standards*; D008007:Licensure, Medical