Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation.
OBJECTIVE
Global assessment by direct observation has been validated for evaluating operative performance of surgery residents after formal skills training but is time-consuming. The purpose of this study was to compare global assessment performed from edited videotape with scores from direct observation.


METHODS
Junior surgery residents (N = 22) were randomized to 2 weeks of formal videotrainer skills training or a control group. Laparoscopic cholecystectomy was performed at the beginning and end of the rotation, and global assessment scores were compared for the training and control groups. Laparoscopic videotapes were edited: initial (2 minutes), cystic duct/artery (6 minutes), and fossa dissection (2 minutes). Two independent raters performed both direct observation and videotape assessments, and scores were compared for each rater and for interrater reliability using a Spearman correlation.


RESULTS
Correlation coefficients for videotape versus direct observation for five global assessment criteria were <0.33 for both raters (NS for all values). The correlation coefficient for interrater reliability for the overall score was 0.57 (P = 0.01) for direct observation v 0.28 (NS) for videotape. The trained group had significantly better overall performance than the control group according to the assessment by direct observation (P = 0.02) but not by videotape assessment (NS).


CONCLUSIONS
Direct observation demonstrated improved overall performance of junior residents after formal skills training on a videotrainer. Global assessment from an edited 10-minute videotape did not correlate with direct observation and had poor interrater reliability. Efficient and valid methods of evaluating operative performance await development.
Scott
D J
DJ
Department of Surgery, The University of Texas Southwestern Medical Center, Dallas, USA.
Rege
R V
RV
Bergen
P C
PC
Guo
W A
WA
Laycock
R
R
Tesfay
S T
ST
Valentine
R J
RJ
Jones
D B
DB
D017081:Cholecystectomy, Laparoscopic; D002983:Clinical Competence; D006801:Humans; D014741:Video Recording