title: Tree-structured CRF models for interactive image labeling.
abstract: We propose structured prediction models for image labeling that explicitly take into account dependencies among image labels. In our tree-structured models, image labels are nodes, and edges encode dependency relations. To allow for more complex dependencies, we combine labels in a single node and use mixtures of trees. Our models are more expressive than independent predictors, and lead to more accurate label predictions. The gain becomes more significant in an interactive scenario where a user provides the value of some of the image labels at test time. Such an interactive scenario offers an interesting tradeoff between label accuracy and manual labeling effort. The structured models are used to decide which labels should be set by the user, and transfer the user input to more accurate predictions on other image labels. We also apply our models to attribute-based image classification, where attribute predictions of a test image are mapped to class probabilities by means of a given attribute-class mapping. Experimental results on three publicly available benchmark datasets show that in all scenarios our structured models lead to more accurate predictions, and leverage user input much more effectively than state-of-the-art independent models.
authors.affiliation: LEAR Team, INRIA Rhone-Alpes, 655 Avenue de l'Europe, Montbonnot 38330, France. thomas.mensink@inria.fr
authors.forename: Thomas
authors.initials: T
authors.lastname: Mensink
authors.forename: Jakob
authors.initials: J
authors.lastname: Verbeek
authors.forename: Gabriela
authors.initials: G
authors.lastname: Csurka
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D004282:Documentation / Q000379:methods*; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D011873:Radiology Information Systems; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014584:User-Computer Interface