title: Alignment of continuous video onto 3D point clouds.
abstract: We propose a general framework for aligning continuous (oblique) video onto 3D sensor data. We align a point cloud computed from the video onto the point cloud directly obtained from a 3D sensor. This is in contrast to existing techniques where the 2D images are aligned to a 3D model derived from the 3D sensor data. Using point clouds enables the alignment for scenes full of objects that are difficult to model; for example, trees. To compute 3D point clouds from video, motion stereo is used along with a state-of-the-art algorithm for camera pose estimation. Our experiments with real data demonstrate the advantages of the proposed registration algorithm for texturing models in large-scale semiurban environments. The capability to align video before a 3D model is built from the 3D sensor data offers new practical opportunities for 3D modeling. We introduce a novel modeling-through-registration approach that fuses 3D information from both the 3D sensor and the video. Initial experiments with real data illustrate the potential of the proposed approach.
authors.affiliation: Vision Technologies Lab, Sarnoff Corporation, 201 Washington Rd., Princeton, NJ 08540, USA. wzhao@sarnoff.com
authors.forename: Wenyi
authors.initials: W
authors.lastname: Zhao
authors.forename: David
authors.initials: D
authors.lastname: Nister
authors.forename: Steve
authors.initials: S
authors.lastname: Hsu
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D016000:Cluster Analysis; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods; D010363:Pattern Recognition, Automated / Q000379:methods*; D010780:Photogrammetry / Q000379:methods*; D013382:Subtraction Technique; D014741:Video Recording / Q000379:methods*