title: Full-frame video stabilization with motion inpainting.
abstract: Video stabilization is an important video enhancement technology which aims at removing annoying shaky motion from videos. We propose a practical and robust approach of video stabilization that produces full-frame stabilized videos with good visual quality. While most previous methods end up with producing smaller size stabilized videos, our completion method can produce full-frame videos by naturally filling in missing image parts by locally aligning image data of neighboring frames. To achieve this, motion inpainting is proposed to enforce spatial and temporal consistency of the completion in both static and dynamic image areas. In addition, image quality in the stabilized video is enhanced with a new practical deblurring algorithm. Instead of estimating point spread functions, our method transfers and interpolates sharper image pixels of neighboring frames to increase the sharpness of the frame. The proposed video completion and deblurring methods enabled us to develop a complete video stabilizer which can naturally keep the original image quality in the stabilized videos. The effectiveness of our method is confirmed by extensive experiments over a wide variety of videos.
authors.affiliation: Microsoft Research Asia, 3/F, Beijing Sigma Center, No. 49, Zhichun Road, Hai Dian District, Beijing, China 100080. yasumat@microsoft.com
authors.forename: Yasuyuki
authors.initials: Y
authors.lastname: Matsushita
authors.forename: Eyal
authors.initials: E
authors.lastname: Ofek
authors.forename: Weina
authors.initials: W
authors.lastname: Ge
authors.forename: Xiaoou
authors.initials: X
authors.lastname: Tang
authors.forename: Heung-Yeung
authors.initials: HY
authors.lastname: Shum
mesh_terms: D000465:Algorithms; D016477:Artifacts; D003196:Computer Graphics; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods*; D009038:Motion; D009716:Numerical Analysis, Computer-Assisted; D010151:Paintings; D010781:Photography / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted; D013382:Subtraction Technique; D014741:Video Recording / Q000379:methods*