title: Minimum cross-entropy pattern classification and cluster analysis.
abstract: This paper considers the problem of classifying an input vector of measurements by a nearest neighbor rule applied to a fixed set of vectors. The fixed vectors are sometimes called characteristic feature vectors, codewords, cluster centers, models, reproductions, etc. The nearest neighbor rule considered uses a non-Euclidean information-theoretic distortion measure that is not a metric, but that nevertheless leads to a classification method that is optimal in a well-defined sense and is also computationally attractive. Furthermore, the distortion measure results in a simple method of computing cluster centroids. Our approach is based on the minimization of cross-entropy (also called discrimination information, directed divergence, K-L number), and can be viewed as a refinement of a general classification method due to Kullback. The refinement exploits special properties of cross-entropy that hold when the probability densities involved happen to be minimum cross-entropy densities. The approach is a generalization of a recently developed speech coding technique called speech coding by vector quantization.
authors.affiliation: SENIOR MEMBER, IEEE, Information Technology Division, Naval Research Laboratory, Washington, DC 20375.
authors.forename: J E
authors.initials: JE
authors.lastname: Shore
authors.forename: R M
authors.initials: RM
authors.lastname: Gray