title: Spatiotemporal inpainting for recovering texture maps of occluded building facades.
abstract: We present a technique for constructing a "clean" texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper, we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses appearance and motion cues in order to fill the texture map in majority-occluded regions. We describe methods for automatically training appearance-based classifiers from a coarse motion-based segmentation to efficiently recognize foreground and background patches in static imagery. Results of recovered building facades are shown for various sequences.
authors.affiliation: Department of Computer and Information Sciences, University of Delaware, Newark, DE 19716, USA. thkorah@udel.edu
authors.forename: Thommen
authors.initials: T
authors.lastname: Korah
authors.forename: Christopher
authors.initials: C
authors.lastname: Rasmussen
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003196:Computer Graphics; D005160:Facility Design and Construction / Q000379:methods*; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000379:methods*; D009716:Numerical Analysis, Computer-Assisted; D010151:Paintings; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity