title: Automatic detection and recognition of signs from natural scenes.
abstract: In this paper, we present an approach to automatic detection and recognition of signs from natural scenes, and its application to a sign translation task. The proposed approach embeds multiresolution and multiscale edge detection, adaptive searching, color analysis, and affine rectification in a hierarchical framework for sign detection, with different emphases at each phase to handle the text in different sizes, orientations, color distributions and backgrounds. We use affine rectification to recover deformation of the text regions caused by an inappropriate camera view angle. The procedure can significantly improve text detection rate and optical character recognition (OCR) accuracy. Instead of using binary information for OCR, we extract features from an intensity image directly. We propose a local intensity normalization method to effectively handle lighting variations, followed by a Gabor transform to obtain local features, and finally a linear discriminant analysis (LDA) method for feature selection. We have applied the approach in developing a Chinese sign translation system, which can automatically detect and recognize Chinese signs as input from a camera, and translate the recognized text into English.
authors.affiliation: School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213 USA. xlchen@cs.cmu.edu
authors.forename: Xilin
authors.initials: X
authors.lastname: Chen
authors.forename: Jie
authors.initials: J
authors.lastname: Yang
authors.forename: Jing
authors.initials: J
authors.lastname: Zhang
authors.forename: Alex
authors.initials: A
authors.lastname: Waibel
mesh_terms: D000465:Algorithms; D001330:Electronic Data Processing / Q000379:methods*; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods*; D008123:Location Directories and Signs; D010363:Pattern Recognition, Automated; D015203:Reproducibility of Results; D012371:Robotics / Q000379:methods; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted