title: Use of an Objective Structured Assessment of Technical Skill After a Sports Medicine Rotation.
abstract: The purpose of this study was to determine if the use of an Objective Structured Assessment of Technical skill (OSATS), using dry models, would be a valid method of assessing residents' ability to perform sports medicine procedures after training in a competency-based model.
  
  
  
  Over 18 months, 27 residents (19 junior [postgraduate year (PGY) 1-3] and 8 senior [PGY 4-5]) sat the OSATS after their rotation, in addition to 14 sports medicine staff and fellows. Each resident was provided a list of 10 procedures in which they were expected to show competence. At the end of the rotation, each resident undertook an OSATS composed of 6 stations sampled from the 10 procedures using dry models-faculty used the Arthroscopic Surgical Skill Evaluation Tool (ASSET), task-specific checklists, as well as an overall 5-point global rating scale (GRS) to score each resident. Each procedure was videotaped for blinded review.
  
  
  
  The overall reliability of the OSATS (0.9) and the inter-rater reliability (0.9) were both high. A significant difference by year in training was seen for the overall GRS, the total ASSET score, and the total checklist score, as well as for each technical procedure (P < .001). Further analysis revealed a significant difference in the total ASSET score between junior (mean 18.4, 95% confidence interval [CI] 16.8 to 19.9) and senior residents (24.2, 95% CI 22.7 to 25.6), senior residents and fellows (30.1, 95% CI 28.2 to 31.9), as well as between fellows and faculty (37, 95% CI 36.1 to 27.8) (P < .05).
  
  
  
  The results of this study show that an OSATS using dry models shows evidence of validity when used to assess performance of technical procedures after a sports medicine rotation. However, junior residents were not able to perform as well as senior residents, suggesting that overall surgical experience is as important as intensive teaching.
  
  
  
  As postgraduate medical training shifts to a competency-based model, methods of assessing performance of technical procedures become necessary.
authors.affiliation: Women's College Hospital, Toronto, Ontario, Canada; Mt. Sinai Hospital, Toronto, Ontario, Canada. Electronic address: tim.dwyer@wchospital.ca.
authors.forename: Tim
authors.initials: T
authors.lastname: Dwyer
authors.affiliation: University of Toronto, Toronto, Ontario, Canada.
authors.forename: Jesse
authors.initials: J
authors.lastname: Slade Shantz
authors.affiliation: The Wilson Centre, Toronto, Ontario, Canada.
authors.forename: Kulamakan Mahan
authors.initials: KM
authors.lastname: Kulasegaram
authors.affiliation: Women's College Hospital, Toronto, Ontario, Canada.
authors.forename: Jaskarndip
authors.initials: J
authors.lastname: Chahal
authors.affiliation: Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.
authors.forename: David
authors.initials: D
authors.lastname: Wasserstein
authors.affiliation: University of Toronto, Toronto, Ontario, Canada.
authors.forename: Rachel
authors.initials: R
authors.lastname: Schachar
authors.affiliation: University of Toronto, Toronto, Ontario, Canada.
authors.forename: Brian
authors.initials: B
authors.lastname: Devitt
authors.affiliation: Women's College Hospital, Toronto, Ontario, Canada; Mt. Sinai Hospital, Toronto, Ontario, Canada.
authors.forename: John
authors.initials: J
authors.lastname: Theodoropoulos
authors.affiliation: The Wilson Centre, Toronto, Ontario, Canada.
authors.forename: Brian
authors.initials: B
authors.lastname: Hodges
authors.affiliation: Women's College Hospital, Toronto, Ontario, Canada.
authors.forename: Darrell
authors.initials: D
authors.lastname: Ogilvie-Harris
mesh_terms: D001182:Arthroscopy / Q000193:education; D057189:Checklist; D002983:Clinical Competence; D006801:Humans; D007396:Internship and Residency; D019637:Orthopedic Procedures / Q000193:education*; D009985:Orthopedics / Q000193:education*; D011446:Prospective Studies; D015203:Reproducibility of Results; D013178:Sports Medicine / Q000193:education*; D014743:Videotape Recording