title: On orthopedic surgical skill prediction--the limited value of traditional testing.
abstract: OBJECTIVE
  Primary: to assess the utility of our distal radius fracture repair model as a tool for examining residents' surgical skills. Secondary: to compare the residents' ability to achieve specific biomechanically measured fracture stability with traditional test scores.
  
  
  METHODS
  Our laboratory pioneered a model that measures biomechanical qualities of a repaired distal radius fracture. Before participation, all residents to be tested completed specified knowledge examinations. During the laboratory exercise, proctors observed each resident and completed Objective Structured Assessment of Technical Skills forms. At the completion of the laboratory, each specimen was tested biomechanically. Written examinations were completed in a proctored setting and computer examinations at home following the honor system. The laboratory exercise had adequate space and materials and allowed 60 minutes to complete the procedure. Residents had equal access to x-ray imaging.
  
  
  METHODS
  The examination environment of the study resembled an operating room.
  
  
  METHODS
  Postgraduate years 3 and 4 orthopedic residents in our program were asked to participate. The institutional review board reviewed and approved the study as exempt.
  
  
  RESULTS
  Fracture repair constructs capable of resisting loads expected during rehabilitation were created by approximately half the residents tested. However, traditional written and computer-based testing methods failed to predict which resident's fracture construct would pass the biomechanical testing. Prior in vivo similar case experience was not predictive.
  
  
  CONCLUSIONS
  The idea that "book smart does not equal street smart" applies to the tested model. To measure surgical skill acquisition and increase public safety related to surgery, it will be necessary to employ new and specific examination methods that identify the skill to be acquired and test the acquisition of this skill as precisely as possible.
authors.affiliation: Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota. Electronic address: mdpmd@umn.edu.
authors.forename: Matthew D
authors.initials: MD
authors.lastname: Putnam
authors.affiliation: Orthopedics, Kaiser Permanente, Roseville, California.
authors.forename: Elspeth
authors.initials: E
authors.lastname: Kinnucan
authors.affiliation: Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota.
authors.forename: Julie E
authors.initials: JE
authors.lastname: Adams
authors.affiliation: Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, Minnesota.
authors.forename: Ann E
authors.initials: AE
authors.lastname: Van Heest
authors.affiliation: Zimmer Spine, Minneapolis, Minnesota.
authors.forename: David J
authors.initials: DJ
authors.lastname: Nuckley
authors.affiliation: Academic Health Center, University of Minnesota, Minneapolis, Minnesota.
authors.forename: Janet
authors.initials: J
authors.lastname: Shanedling
mesh_terms: D001696:Biomechanical Phenomena; D002102:Cadaver; D002983:Clinical Competence; D003198:Computer Simulation; D004503:Education, Medical, Graduate / Q000458:organization & administration*; D004521:Educational Measurement / Q000379:methods*; D005592:Fracture Fixation / Q000193:education* / Q000379:methods; D006801:Humans; D020407:Internet; D007396:Internship and Residency; D008910:Minnesota; D009985:Orthopedics / Q000193:education*; D010043:Outcome and Process Assessment, Health Care; D011885:Radius Fractures / Q000601:surgery*
keywords: Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; criterion-based skill testing; resident education; standardized; surgical competence