title: Automatic eyeglasses removal from face images.
abstract: In this paper, we present an intelligent image editing and face synthesis system that automatically removes eyeglasses from an input frontal face image. Although conventional image editing tools can be used to remove eyeglasses by pixel-level editing, filling in the deleted eyeglasses region with the right content is a difficult problem. Our approach works at the object level where the eyeglasses are automatically located, removed as one piece, and the void region filled. Our system consists of three parts: eyeglasses detection, eyeglasses localization, and eyeglasses removal. First, an eye region detector, trained offline, is used to approximately locate the region of eyes, thus the region of eyeglasses. A Markov-chain Monte Carlo method is then used to accurately locate key points on the eyeglasses frame by searching for the global optimum of the posterior. Subsequently, a novel sample-based approach is used to synthesize the face image without the eyeglasses. Specifically, we adopt a statistical analysis and synthesis approach to learn the mapping between pairs of face images with and without eyeglasses from a database. Extensive experiments demonstrate that our system effectively removes eyeglasses.
authors.affiliation: Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA. chenyuwu@cmu.edu
authors.forename: Chenyu
authors.initials: C
authors.lastname: Wu
authors.forename: Ce
authors.initials: C
authors.lastname: Liu
authors.forename: Heung-Yueng
authors.initials: HY
authors.lastname: Shum
authors.forename: Ying-Qing
authors.initials: YQ
authors.lastname: Xu
authors.forename: Zhengyou
authors.initials: Z
authors.lastname: Zhang
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D005139:Eyeglasses; D005145:Face / Q000033:anatomy & histology*; D005260:Female; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval; D008297:Male; D010363:Pattern Recognition, Automated; D010781:Photography / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted; D013382:Subtraction Technique; D014584:User-Computer Interface