Video-based assessment of operative competency in endoscopic sinus surgery.
BACKGROUND
The Accreditation Council for Graduate Medical Education requires that residency programs have an effective plan for assessing their residents' competence. Current evaluation tools used for the assessment of operative competency are subjective, recall based, and do not allow providing formative feedback to the resident. The purpose of this study was to evaluate the feasibility and validity of video-based assessment of endoscopic sinus surgery (ESS) skills.


METHODS
Five junior (postgraduate year levels 1-3[PGY-1 to -3]) and three senior (PGY-4 and -5) residents were videotaped as they performed ESS on cadavers in the laboratory. Five faculty members watched the recorded videotapes and rated performance using global and checklist assessment tools for ESS. Evaluators were blinded to the identity of the performers. Rating was made feasible by allowing the raters to fast-forward through the tapes when appropriate. Construct validity was calculated by comparing total scores and score on each item of the instrument across the training levels. For all statistical purposes p < 0.05 was considered significant.


RESULTS
Results showed construct validity with senior residents performing better than junior residents. It took an average of 20 minutes (range, 7- 39 minutes) to watch and evaluate a video. Interrater reliability, as measured by interclass correlation coefficient across evaluators, was 0.62.


CONCLUSIONS
The use of videotapes combined with the objective assessment tool is a feasible and valid method for evaluating surgical skills. It has implications for residency programs, because it will increase faculty buy-in and reduce the potential for biased assessment.
Laeeq
Kulsoom
K
Department of Otolaryngology-Head and Neck Surgery, The Johns Hopkins University, School of Medicine, 601 N. Caroline Street, Baltimore, MD 21287, USA.
Infusino
Scott
S
Lin
Sandra Y
SY
Reh
Douglas D
DD
Ishii
Masaru
M
Kim
Jean
J
Lane
Andrew P
AP
Bhatti
Nasir I
NI
D002102:Cadaver; D002908:Chronic Disease; D002983:Clinical Competence; D004503:Education, Medical, Graduate; D004521:Educational Measurement; D004724:Endoscopy; D005240:Feasibility Studies; D006801:Humans; D007396:Internship and Residency / Q000379:methods; D012220:Rhinitis / Q000150:complications / Q000175:diagnosis / Q000473:pathology / Q000503:physiopathology / Q000601:surgery*; D012852:Sinusitis / Q000150:complications / Q000175:diagnosis / Q000473:pathology / Q000503:physiopathology / Q000601:surgery*; D012984:Software; D014481:United States; D020535:Video-Assisted Surgery