title: Continuous, data-rich appraisal of surgical trainees' operative abilities: a novel approach for measuring performance and providing feedback.
abstract: BACKGROUND
  We developed a convenient mechanism, Surgical Training and Assessment Tool (STAT), to accomplish detailed, continuous analysis of surgical trainees' operative abilities, and a simple method, Quality Based Surgical Training (QBST) for implementing it.
  
  
  METHODS
  Using a web-accessed computer program, attending physicians and trainees independently assessed the trainee's operative performance after every operative (training) case. Global attributes of surgical knowledge, skill, and independence were assessed as well as the key technical maneuvers of each operation. A system of hierarchical, expandable menus specific to each of hundreds of different surgical procedures allowed the assessments to be made as detailed or as general as the users felt were necessary. In addition, freehand, unscripted commentary was recorded via an optional "remarks" box feature. Finally, an independently chosen, "overall" grade scaled F through A+ concluded each assessment.
  
  
  RESULTS
  Over a 31 month period, 72 different users (52 trainees, 20 attending physicians) submitted 3849 performance assessments on 2424 cases, including 132 different case types and amassing 68,260 distinct data points. The mean number of data points per trainee was 1313; the median time spent per assessment was 60 seconds. Graphic displays allowed formative review of individual cases in real time, and summative review of long term trends. Appraisals of knowledge, skill, and independence were strongly correlated with and independently predictive of the overall competency grade (model r(2) = 0.68; test of predictive significance p < 0.001 for each rating). Trainee and attending physician scores were highly correlated (> 0.7) with one another.
  
  
  CONCLUSIONS
  QBST/STAT achieves detailed, continuous analysis of surgical trainees' operative abilities, and facilitates timely, specific, and thorough feedback regarding their performance in theater. QBST/STAT promotes trainee self-reflection and generation of continuous, transparent, iterative training goals.
authors.affiliation: Department of Surgery, Naval Medical Center Portsmouth, Portsmouth, Virginia.
authors.forename: Paul B
authors.initials: PB
authors.lastname: Roach
authors.forename: Kevin K
authors.initials: KK
authors.lastname: Roggin
authors.forename: Gene
authors.initials: G
authors.lastname: Selkov
authors.forename: Mitchell C
authors.initials: MC
authors.lastname: Posner
authors.forename: Jonathan C
authors.initials: JC
authors.lastname: Silverstein
mesh_terms: D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005246:Feedback; D005260:Female; D006801:Humans; D007396:Internship and Residency; D008297:Male; D016347:Medical Records Systems, Computerized; D008505:Medical Staff, Hospital; D011787:Quality of Health Care; D013043:Specialties, Surgical / Q000193:education*; D013514:Surgical Procedures, Operative / Q000193:education*; D013647:Task Performance and Analysis; D014481:United States