title: Generalized face super-resolution.
abstract: Existing learning-based face super-resolution (hallucination) techniques generate high-resolution images of a single facial modality (i.e., at a fixed expression, pose and illumination) given one or set of low-resolution face images as probe. Here, we present a generalized approach based on a hierarchical tensor (multilinear) space representation for hallucinating high-resolution face images across multiple modalities, achieving generalization to variations in expression and pose. In particular, we formulate a unified tensor which can be reduced to two parts: a global image-based tensor for modeling the mappings among different facial modalities, and a local patch-based multiresolution tensor for incorporating high-resolution image details. For realistic hallucination of unregistered low-resolution faces contained in raw images, we develop an automatic face alignment algorithm capable of pixel-wise alignment by iteratively warping the probing face to its projection in the space of training face images. Our experiments show not only performance superiority over existing benchmark face super-resolution techniques on single modal face hallucination, but also novelty of our approach in coping with multimodal hallucination and its robustness in automatic alignment under practical imaging conditions.
authors.affiliation: Shenzhen Institute of Advanced Integration Technology, Chinese Academy of Sciences/Chinese Academy of Hong Kong, Shenzhen, China. chrisjia@dcs.qmul.ac.uk
authors.forename: Kui
authors.initials: K
authors.lastname: Jia
authors.forename: Shaogang
authors.initials: S
authors.lastname: Gong
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001699:Biometry / Q000379:methods*; D005145:Face / Q000033:anatomy & histology*; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013382:Subtraction Technique