title: Learning with hierarchical-deep models.
abstract: We introduce HD (or &#x201C;Hierarchical-Deep&#x201D;) models, a new compositional learning architecture that integrates deep learning models with structured hierarchical Bayesian (HB) models. Specifically, we show how we can learn a hierarchical Dirichlet process (HDP) prior over the activities of the top-level features in a deep Boltzmann machine (DBM). This compound HDP-DBM model learns to learn novel concepts from very few training example by learning low-level generic features, high-level features that capture correlations among low-level features, and a category hierarchy for sharing priors over the high-level features that are typical of different kinds of concepts. We present efficient learning and inference algorithms for the HDP-DBM model and show that it is able to learn new concepts from very few examples on CIFAR-100 object recognition, handwritten character recognition, and human motion capture datasets.
authors.affiliation: Department of Statistics and Computer Science, University of Toronto, Toronto, ON M5S 3G3, Canada. rsalakhu@utstat.toronto.edu
authors.forename: Ruslan
authors.initials: R
authors.lastname: Salakhutdinov
authors.forename: Joshua B
authors.initials: JB
authors.lastname: Tenenbaum
authors.forename: Antonio
authors.initials: A
authors.lastname: Torralba
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001499:Bayes Theorem; D006801:Humans; D009038:Motion; D010363:Pattern Recognition, Automated / Q000379:methods*; D014796:Visual Perception