title: Cross-modal learning to rank via latent joint representation.
abstract: Cross-modal ranking is a research topic that is imperative to many applications involving multimodal data. Discovering a joint representation for multimodal data and learning a ranking function are essential in order to boost the cross-media retrieval (i.e., image-query-text or text-query-image). In this paper, we propose an approach to discover the latent joint representation of pairs of multimodal data (e.g., pairs of an image query and a text document) via a conditional random field and structural learning in a listwise ranking manner. We call this approach cross-modal learning to rank via latent joint representation (CML²R). In CML²R, the correlations between multimodal data are captured in terms of their sharing hidden variables (e.g., topics), and a hidden-topic-driven discriminative ranking function is learned in a listwise ranking manner. The experiments show that the proposed approach achieves a good performance in cross-media retrieval and meanwhile has the capability to learn the discriminative representation of multimodal data.
authors.forename: Fei
authors.initials: F
authors.lastname: Wu
authors.forename: Xinyang
authors.initials: X
authors.lastname: Jiang
authors.forename: Xi
authors.initials: X
authors.lastname: Li
authors.forename: Siliang
authors.initials: S
authors.lastname: Tang
authors.forename: Weiming
authors.initials: W
authors.lastname: Lu
authors.forename: Zhongfei
authors.initials: Z
authors.lastname: Zhang
authors.forename: Yueting
authors.initials: Y
authors.lastname: Zhuang