title: A Variational Approach to Video Registration with Subspace Constraints.
abstract: This paper addresses the problem of non-rigid video registration, or the computation of optical flow from a reference frame to each of the subsequent images in a sequence, when the camera views deformable objects. We exploit the high correlation between 2D trajectories of different points on the same non-rigid surface by assuming that the displacement of any point throughout the sequence can be expressed in a compact way as a linear combination of a low-rank motion basis. This subspace constraint effectively acts as a trajectory regularization term leading to temporally consistent optical flow. We formulate it as a robust soft constraint within a variational framework by penalizing flow fields that lie outside the low-rank manifold. The resulting energy functional can be decoupled into the optimization of the brightness constancy and spatial regularization terms, leading to an efficient optimization scheme. Additionally, we propose a novel optimization scheme for the case of vector valued images, based on the dualization of the data term. This allows us to extend our approach to deal with colour images which results in significant improvements on the registration results. Finally, we provide a new benchmark dataset, based on motion capture data of a flag waving in the wind, with dense ground truth optical flow for evaluation of multi-frame optical flow algorithms for non-rigid surfaces. Our experiments show that our proposed approach outperforms state of the art optical flow and dense non-rigid registration algorithms.
authors.affiliation: Queen Mary University of London, Mile End Road, London, E1 4NS UK.
authors.forename: Ravi
authors.initials: R
authors.lastname: Garg
authors.forename: Anastasios
authors.initials: A
authors.lastname: Roussos
authors.forename: Lourdes
authors.initials: L
authors.lastname: Agapito
references.citation: IEEE Trans Image Process. 1996;5(4):598-610
references.pmid: 18285148
references.citation: IEEE Trans Pattern Anal Mach Intell. 2011 Jul;33(7):1442-56
references.pmid: 21079275
references.citation: IEEE Trans Pattern Anal Mach Intell. 2011 Mar;33(3):500-13
references.pmid: 20714020
references.citation: IEEE Trans Image Process. 1998;7(3):304-9
references.pmid: 18276250
references.citation: IEEE Trans Pattern Anal Mach Intell. 2011 May;33(5):978-94
references.pmid: 20714019
references.citation: IEEE Trans Pattern Anal Mach Intell. 2008 May;30(5):878-92
references.pmid: 18369256
references.citation: IEEE Trans Pattern Anal Mach Intell. 2005 Apr;27(4):506-517
references.pmid: 15794157