title: Medical Image Retrieval: A Multimodal Approach.
abstract: Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.
authors: Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.
authors: Yu
authors: Y
authors: Cao
authors: Department of Computer Science, The University of Massachusetts Lowell, Lowell, MA, USA.
authors: Shawn
authors: S
authors: Steffey
authors: School of Information Science and Engineering, Central South University, Changsha, PR China.
authors: Jianbiao
authors: J
authors: He
authors: College of Computer Science and Electronic Engineering, Hunan University, Changsha, PR China.
authors: Degui
authors: D
authors: Xiao
authors: School of Biomedical Informatics, The University of Texas, Health Science Center at Houston, Houston, TX, USA.
authors: Cui
authors: C
authors: Tao
authors: Department of Computer Science, University of Massachusetts Boston, Boston, MA, USA.
authors: Ping
authors: P
authors: Chen
authors: Department of Business Information Systems, University of Applied Sciences Western Switzerland (HES-SO), Medical Informatics, University Hospitals and University of Geneva, Geneva, Switzerland.
authors: Henning
authors: H
authors: Müller
keywords: content-based image retrieval; deep boltzmann machine; deep learning; extended probabilistic latent semantic analysis; multi-modal and content-based medical image retrieval