title: Free-viewpoint video of human actors using multiple handheld Kinects.
abstract: We present an algorithm for creating free-viewpoint video of interacting humans using three handheld Kinect cameras. Our method reconstructs deforming surface geometry and temporal varying texture of humans through estimation of human poses and camera poses for every time step of the RGBZ video. Skeletal configurations and camera poses are found by solving a joint energy minimization problem, which optimizes the alignment of RGBZ data from all cameras, as well as the alignment of human shape templates to the Kinect data. The energy function is based on a combination of geometric correspondence finding, implicit scene segmentation, and correspondence finding using image features. Finally, texture recovery is achieved through jointly optimization on spatio-temporal RGB data using matrix completion. As opposed to previous methods, our algorithm succeeds on free-viewpoint video of human actors under general uncontrolled indoor scenes with potentially dynamic background, and it succeeds even if the cameras are moving.
authors.forename: Genzhi
authors.initials: G
authors.lastname: Ye
authors.forename: Yebin
authors.initials: Y
authors.lastname: Liu
authors.forename: Yue
authors.initials: Y
authors.lastname: Deng
authors.forename: Nils
authors.initials: N
authors.lastname: Hasler
authors.forename: Xiangyang
authors.initials: X
authors.lastname: Ji
authors.forename: Qionghai
authors.initials: Q
authors.lastname: Dai
authors.forename: Christian
authors.initials: C
authors.lastname: Theobalt
mesh_terms: D056044:Actigraphy / Q000295:instrumentation / Q000379:methods*; D000465:Algorithms; D001185:Artificial Intelligence; D016287:Computer Peripherals; D003198:Computer Simulation; D006801:Humans; D007089:Image Enhancement / Q000295:instrumentation / Q000379:methods; D021621:Imaging, Three-Dimensional / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D014159:Transducers; D018910:Video Games; D014741:Video Recording / Q000379:methods*; D051598:Whole Body Imaging / Q000295:instrumentation / Q000379:methods*