title: Real-time recognition of surgical tasks in eye surgery videos.
abstract: Nowadays, many surgeries, including eye surgeries, are video-monitored. We present in this paper an automatic video analysis system able to recognize surgical tasks in real-time. The proposed system relies on the Content-Based Video Retrieval (CBVR) paradigm. It characterizes short subsequences in the video stream and searches for video subsequences with similar structures in a video archive. Fixed-length feature vectors are built for each subsequence: the feature vectors are unchanged by variations in duration and temporal structure among the target surgical tasks. Therefore, it is possible to perform fast nearest neighbor searches in the video archive. The retrieved video subsequences are used to recognize the current surgical task by analogy reasoning. The system can be trained to recognize any surgical task using weak annotations only. It was applied to a dataset of 23 epiretinal membrane surgeries and a dataset of 100 cataract surgeries. Three surgical tasks were annotated in the first dataset. Nine surgical tasks were annotated in the second dataset. To assess its generality, the system was also applied to a dataset of 1,707 movie clips in which 12 human actions were annotated. High task recognition scores were measured in all three datasets. Real-time task recognition will be used in future works to communicate with surgeons (trainees in particular) or with surgical devices.
authors.affiliation: Inserm, UMR 1101, Brest F-29200, France. Electronic address: gwenole.quellec@inserm.fr.
authors.forename: Gwénolé
authors.initials: G
authors.lastname: Quellec
authors.affiliation: INSTITUT Mines-Télécom, TELECOM Bretagne, UEB, Dpt ITI, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France.
authors.forename: Katia
authors.initials: K
authors.lastname: Charrière
authors.affiliation: Univ Bretagne Occidentale, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France.
authors.forename: Mathieu
authors.initials: M
authors.lastname: Lamard
authors.affiliation: INSTITUT Mines-Télécom, TELECOM Bretagne, UEB, Dpt ITI, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France.
authors.forename: Zakarya
authors.initials: Z
authors.lastname: Droueche
authors.affiliation: INSTITUT Mines-Télécom, TELECOM Bretagne, UEB, Dpt ITI, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France.
authors.forename: Christian
authors.initials: C
authors.lastname: Roux
authors.affiliation: Univ Bretagne Occidentale, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France; CHRU Brest, Service d'Ophtalmologie, Brest F-29200, France.
authors.forename: Béatrice
authors.initials: B
authors.lastname: Cochener
authors.affiliation: INSTITUT Mines-Télécom, TELECOM Bretagne, UEB, Dpt ITI, Brest F-29200, France; Inserm, UMR 1101, Brest F-29200, France.
authors.forename: Guy
authors.initials: G
authors.lastname: Cazuguel
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003199:Computer Systems; D005128:Eye Diseases / Q000601:surgery; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000379:methods*; D013508:Ophthalmologic Surgical Procedures / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D010781:Photography / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D025321:Surgery, Computer-Assisted / Q000379:methods*; D014741:Video Recording / Q000379:methods*
keywords: CBVR; Eye surgery; Real-time; Surgical task recognition