title: Relationship among United States Medical Licensing Step I, orthopedic in-training, subjective clinical performance evaluations, and american board of orthopedic surgery examination scores: a 12-year review of an orthopedic surgery residency program.
abstract: OBJECTIVE
  To improve the understanding of relationships among United States Medical Licensing Examination (USMLE Step I), Orthopedic In-Training Examination (OITE), Subjective Clinical Performance Evaluations, and American Board of Orthopedic Surgery Examination Part I (Abos-I) and Part II (Abos-II), which would help residency programs better achieve their educational mission.
  
  
  METHODS
  A 12-year descriptive study of retrospectively collected data.
  
  
  METHODS
  One residency program with 47 resident participants.
  
  
  RESULTS
  Residents that failed Abos-I and Abos-II had lower program mean OITE year-in-training (YIT) percentile rank scores. The program mean OITE YIT percentile rank score had a moderate relationship with Abos-I (% correct) score (r = 0.68, p < 0.0001) and an insignificant relationship with USMLE Step I (3-digit) score (r = 0.22, p = 0.13). Residents with upper quartile (>or=220) USMLE Step I (3-digit) scores for our program had higher program mean OITE YIT percentile rank scores and Abos-I (% correct) scores than residents with lower quartile scores (<or=202). Residents who scored in the upper quartile (>or=55) for the program mean OITE YIT percentile rank score had higher Abos-I (% correct) scores than residents who did not. Residents who scored in the lower quartile for the third postgraduate year (PGY-3) program OITE YIT percentile rank score or for the program mean OITE YIT percentile rank score had a 5.2 and 5.8 time greater Abos-I failure risk, respectively. The program PGY-3 OITE YIT percentile rank score was the strongest Abos-I (% correct) score discriminator. Resident Abos-I (% correct), program mean OITE YIT, and program PGY-3 OITE YIT percentile rank scores were the strongest discriminators for Abos-II passage. Residents with a program mean OITE YIT percentile rank score >or=28, program PGY-3 OITE YIT percentile rank score >or=39, and USMLE Step I (3-digit) score >or=207 were more likely to pass Abos-I and II. Residents that had lower quartile USMLE Step I (3-digit) scores for our program had a 2.3 time greater Abos-I failure risk. Program residents with >or=2 below-average subjective clinical performance evaluations had lower Abos-I (% correct) scores but had similar Abos-I and II pass rates.
  
  
  CONCLUSIONS
  Our program uses the USMLE Step I (3-digit) score as a preacceptance estimate of likely supplemental guided mentoring needs. Program mean OITE YIT percentile rank and PGY-3 OITE YIT percentile rank scores help identify educational deficiencies and predict eventual Abos-I and II passage. Subjective clinical performance evaluations provide important supplemental information regarding professionalism, communication, and patient care skills.
authors.affiliation: Department of Orthopaedic Surgery, University of Louisville, Louisville, Kentucky 40202, USA.
authors.forename: Charles H
authors.initials: CH
authors.lastname: Crawford
authors.forename: John
authors.initials: J
authors.lastname: Nyland
authors.forename: Craig S
authors.initials: CS
authors.lastname: Roberts
authors.forename: John R
authors.initials: JR
authors.lastname: Johnson
mesh_terms: D000704:Analysis of Variance; D004503:Education, Medical, Graduate; D004521:Educational Measurement; D006801:Humans; D007396:Internship and Residency; D008007:Licensure, Medical; D009985:Orthopedics / Q000193:education*; D012372:ROC Curve; D012189:Retrospective Studies; D012570:School Admission Criteria; D018709:Statistics, Nonparametric; D014481:United States