Validation of video-based skill assessment in carotid artery stenting.
OBJECTIVE
To develop weighted error-based, generic and procedure-specific rating scales, to validate these scales for video-based assessment during virtual carotid artery stent (CAS) procedures and correlate them with simulator-derived metrics.


METHODS
A questionnaire was developed to assess the technique during live CAS procedures. Errors were rated from 1 (unimportant) to 5 (life-threatening) by 28 highly experienced CAS (>50 CAS) physicians. Virtual CAS procedure was performed by 21 interventionalists with varied CAS experience. Fluoroscopy screen and hand movements were video-taped, and simulator-derived metrics recorded. Experienced CAS practitioners then rated the video-taped performances using weighted error, generic and procedure-specific rating scales.


RESULTS
Of the 23 errors assessed, 12 were regarded as moderate (score 3), six serious (score 4) and four life-threatening (score 5). The generic rating scale was able to detect significant differences in performance between inexperienced and experienced CAS operators (score 25 vs. 32 respectively, P<0.01). All scoring systems demonstrated good inter-rater reliability (alpha=0.61-0.87). Significant correlations were observed between simulator-derived and video-based scores: weighted error-based score (r: 0.76, P<0.01), generic (r: 0.62, P<0.01) and procedure-specific (r: 0.76, P<0.01) rating scales.


CONCLUSIONS
The generic endovascular rating scale differentiated between levels of CAS experience among skilled interventionalists and correlated to simulator-based error scoring.
Van Herzeele
I
I
Department of Thoracic and Vascular Surgery, Ghent University Hospital, Gent, Belgium. vhisabelle@gmail.com
Aggarwal
R
R
Malik
I
I
Gaines
P
P
Hamady
M
M
Darzi
A
A
Cheshire
N
N
Vermassen
F
F
D019917:Blood Vessel Prosthesis Implantation / Q000592:standards*; D016893:Carotid Stenosis / Q000601:surgery*; D002983:Clinical Competence / Q000592:standards*; D006801:Humans; D019300:Medical Errors / Q000517:prevention & control* / Q000706:statistics & numerical data; D017063:Outcome Assessment, Health Care / Q000379:methods*; D015607:Stents; D011795:Surveys and Questionnaires; D014584:User-Computer Interface; D014741:Video Recording / Q000379:methods*