title: An operative performance rating system for urology residents.
abstract: OBJECTIVE
  An operative performance rating system for urology residents was developed for 6 sentinel urological procedures. We tested the reliability, validity and feasibility of the operative performance rating system for urology residents.
  
  
  METHODS
  The operative performance rating system of each procedure contained a 3-point case difficulty scale, 4 to 6 procedure specific items, 3 general items and an overall performance item. A Likert scale of 1 to 5 was used for each item. A single video/audio record of each procedure was evaluated by the faculty. Single item interrater agreement was measured by comparing the observed variance and random measurement error variance. Resident operative performance evaluations were completed on line. Internal consistency reliability was measured using Cronbach α. Overall scale scores by resident training postgraduate year level were compared using 1-way ANOVA.
  
  
  RESULTS
  Faculty evaluation of video/audio records showed an interrater agreement range of 0.71 to 0.92. Faculty evaluations of resident operative performance demonstrated an internal consistency reliability range of 0.91 to 0.95. Significant differences in overall scale scores between postgraduate year levels were noted for 3 of the 6 procedures (p ≤0.0016).
  
  
  CONCLUSIONS
  An operative performance rating system for urology residents is feasible using an Internet based resident management system. Interrater agreement and internal consistency reliability meet threshold limits for checklist evaluation instruments. The operative performance rating system can discriminate among postgraduate year levels of resident training. A validated operative performance rating system can offer residents immediate, objective feedback on surgical performance and enable program directors to monitor progress in resident operative performance.
authors.affiliation: Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois, USA.
authors.forename: Aaron
authors.initials: A
authors.lastname: Benson
authors.forename: Stephen
authors.initials: S
authors.lastname: Markwell
authors.forename: Tobias S
authors.initials: TS
authors.lastname: Kohler
authors.forename: Thomas H
authors.initials: TH
authors.lastname: Tarter
mesh_terms: D002983:Clinical Competence; D005240:Feasibility Studies; D006801:Humans; D007396:Internship and Residency; D015203:Reproducibility of Results; D013520:Urologic Surgical Procedures / Q000592:standards*; D014572:Urology / Q000193:education*