title: Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation.
abstract: OBJECTIVE
  Global assessment by direct observation has been validated for evaluating operative performance of surgery residents after formal skills training but is time-consuming. The purpose of this study was to compare global assessment performed from edited videotape with scores from direct observation.
  
  
  METHODS
  Junior surgery residents (N = 22) were randomized to 2 weeks of formal videotrainer skills training or a control group. Laparoscopic cholecystectomy was performed at the beginning and end of the rotation, and global assessment scores were compared for the training and control groups. Laparoscopic videotapes were edited: initial (2 minutes), cystic duct/artery (6 minutes), and fossa dissection (2 minutes). Two independent raters performed both direct observation and videotape assessments, and scores were compared for each rater and for interrater reliability using a Spearman correlation.
  
  
  RESULTS
  Correlation coefficients for videotape versus direct observation for five global assessment criteria were <0.33 for both raters (NS for all values). The correlation coefficient for interrater reliability for the overall score was 0.57 (P = 0.01) for direct observation v 0.28 (NS) for videotape. The trained group had significantly better overall performance than the control group according to the assessment by direct observation (P = 0.02) but not by videotape assessment (NS).
  
  
  CONCLUSIONS
  Direct observation demonstrated improved overall performance of junior residents after formal skills training on a videotrainer. Global assessment from an edited 10-minute videotape did not correlate with direct observation and had poor interrater reliability. Efficient and valid methods of evaluating operative performance await development.
authors.affiliation: Department of Surgery, The University of Texas Southwestern Medical Center, Dallas, USA.
authors.forename: D J
authors.initials: DJ
authors.lastname: Scott
authors.forename: R V
authors.initials: RV
authors.lastname: Rege
authors.forename: P C
authors.initials: PC
authors.lastname: Bergen
authors.forename: W A
authors.initials: WA
authors.lastname: Guo
authors.forename: R
authors.initials: R
authors.lastname: Laycock
authors.forename: S T
authors.initials: ST
authors.lastname: Tesfay
authors.forename: R J
authors.initials: RJ
authors.lastname: Valentine
authors.forename: D B
authors.initials: DB
authors.lastname: Jones
mesh_terms: D017081:Cholecystectomy, Laparoscopic; D002983:Clinical Competence; D006801:Humans; D014741:Video Recording