title: Predicting performance on the Medical Council of Canada Qualifying Exam Part II.
abstract: BACKGROUND
  Being able to predict which residents will likely be unsuccessful on high-stakes exams would allow residency programs to provide early intervention.
  
  
  OBJECTIVE
  To determine whether measures of clinical performance in clerkship (in-training evaluation reports) and first year of residency (program director ratings) predict pass-fail performance on the Medical Council of Canada Qualifying Exam Part II (MCCQE Part II).
  
  
  METHODS
  Residency program directors assessed the performance of our medical school graduates (Classes 2004-2007) at the end of the 1st postgraduate year. We subsequently collected clerkship in-training evaluation reports for these graduates. Using a neutral third party and unique codes, an anonymous dataset containing clerkship, residency, and MCCQE Part II performance scores was created for our use. Data were analyzed using descriptive statistics, correlations, receiver operating characteristics, and the Youdin index. Regression was also performed to further study the relationship among the variables.
  
  
  RESULTS
  Complete data were available for 78.6% of the graduates. Of these participants, 94% passed the licensing exam on their first attempt. Receiver operating characteristics revealed that the area under the curve for clerkship in-training evaluation reports was 0.67 (p<.05) and 0.66 (p<.05) for residency program directors assessments. Corresponding Youdin indices for in-training evaluation reports and residency program director assessments were 0.30 and 0.23, respectively.
  
  
  CONCLUSIONS
  Although clerkship in-training evaluation reports and residency program director ratings are significant predictors of pass-fail performance on the MCCQE Part II, the effectiveness of each one to predict pass-fail performance was relatively small. Reasons for these findings are discussed.
authors.affiliation: Undergraduate Medical Education, University of Calgary, Calgary, Alberta, Canada. woloschu@ucalgary.ca
authors.forename: Wayne
authors.initials: W
authors.lastname: Woloschuk
authors.forename: Kevin
authors.initials: K
authors.lastname: McLaughlin
authors.forename: Bruce
authors.initials: B
authors.lastname: Wright
mesh_terms: D002170:Canada; D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D004522:Educational Status; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D008297:Male; D011237:Predictive Value of Tests