title: Robust color histogram descriptors for video segment retrieval and identification.
abstract: Effective and efficient representation of color features of multiple video frames or pictures is an important yet challenging task for visual information management systems. Key frame-based methods to represent the color features of a group of frames (GoF) are highly dependent on the selection criterion of the representative frame(s), and may lead to unreliable results. We present various histogram-based color descriptors to reliably capture and represent the color properties of multiple images or a GoF. One family of such descriptors, called alpha-trimmed average histograms, combine individual frame or image histograms using a specific filtering operation to generate robust color histograms that can eliminate the adverse effects of brightness/color variations, occlusion, and edit effects on the color representation. We show the efficacy of the alpha-trimmed average histograms for video segment retrieval applications, and illustrate how they consistently outperform key frame-based methods. Another color histogram descriptor that we introduce, called the intersection histogram, reflects the number of pixels of a given color that is common to all the frames in the GoF. We employ the intersection histogram to develop a fast and efficient algorithm for identification of the video segment to which a query frame belongs. The proposed color histogram descriptors have been included in the ISO standard MPEG-7 after extensive evaluation experiments.
authors.affiliation: Department of Electrical and Computer Engineering, University of Rochester, NY 14627-0126, USA. mferman@sharplabs.com
authors.forename: A Müfit
authors.initials: AM
authors.lastname: Ferman
authors.forename: A Murat
authors.initials: AM
authors.lastname: Tekalp
authors.forename: Rajiv
authors.initials: R
authors.lastname: Mehrotra