title: Articulated clinician detection using 3D pictorial structures on RGB-D data.
abstract: Reliable human pose estimation (HPE) is essential to many clinical applications, such as surgical workflow analysis, radiation safety monitoring and human-robot cooperation. Proposed methods for the operating room (OR) rely either on foreground estimation using a multi-camera system, which is a challenge in real ORs due to color similarities and frequent illumination changes, or on wearable sensors or markers, which are invasive and therefore difficult to introduce in the room. Instead, we propose a novel approach based on Pictorial Structures (PS) and on RGB-D data, which can be easily deployed in real ORs. We extend the PS framework in two ways. First, we build robust and discriminative part detectors using both color and depth images. We also present a novel descriptor for depth images, called histogram of depth differences (HDD). Second, we extend PS to 3D by proposing 3D pairwise constraints and a new method that makes exact inference tractable. Our approach is evaluated for pose estimation and clinician detection on a challenging RGB-D dataset recorded in a busy operating room during live surgeries. We conduct series of experiments to study the different part detectors in conjunction with the various 2D or 3D pairwise constraints. Our comparisons demonstrate that 3D PS with RGB-D part detectors significantly improves the results in a visually challenging operating environment.
authors.affiliation: ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France. Electronic address: kadkhodamohammad@unistra.fr.
authors.forename: Abdolrahim
authors.initials: A
authors.lastname: Kadkhodamohammadi
authors.affiliation: ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France; Radiology Department, University Hospital of Strasbourg, Strasbourg, France.
authors.forename: Afshin
authors.initials: A
authors.lastname: Gangi
authors.affiliation: ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France.
authors.forename: Michel
authors.initials: M
authors.lastname: de Mathelin
authors.affiliation: ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France.
authors.forename: Nicolas
authors.initials: N
authors.lastname: Padoy
mesh_terms: D000465:Algorithms; D003116:Color; D013502:General Surgery; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000379:methods*; D009873:Operating Rooms
keywords: 3D Pictorial structures; Clinician pose estimation; Medical computer vision; RGB-D Data; Surgical activity analysis