title: Supervised learning of quantizer codebooks by information loss minimization.
abstract: This paper proposes a technique for jointly quantizing continuous features and the posterior distributions of their class labels based on minimizing empirical information loss such that the quantizer index of a given feature vector approximates a sufficient statistic for its class label. Informally, the quantized representation retains as much information as possible for classifying the feature vector correctly. We derive an alternating minimization procedure for simultaneously learning codebooks in the euclidean feature space and in the simplex of posterior class distributions. The resulting quantizer can be used to encode unlabeled points outside the training set and to predict their posterior class distributions, and has an elegant interpretation in terms of lossless source coding. The proposed method is validated on synthetic and real data sets and is applied to two diverse problems: learning discriminative visual vocabularies for bag-of-features image classification and image segmentation.
authors.affiliation: Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA. lazebnik@cs.unc.edu
authors.forename: Svetlana
authors.initials: S
authors.lastname: Lazebnik
authors.forename: Maxim
authors.initials: M
authors.lastname: Raginsky
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D044962:Data Compression / Q000379:methods*; D009716:Numerical Analysis, Computer-Assisted; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted