title: A graph-based approach for the retrieval of multi-modality medical images.
abstract: In this paper, we address the retrieval of multi-modality medical volumes, which consist of two different imaging modalities, acquired sequentially, from the same scanner. One such example, positron emission tomography and computed tomography (PET-CT), provides physicians with complementary functional and anatomical features as well as spatial relationships and has led to improved cancer diagnosis, localisation, and staging. The challenge of multi-modality volume retrieval for cancer patients lies in representing the complementary geometric and topologic attributes between tumours and organs. These attributes and relationships, which are used for tumour staging and classification, can be formulated as a graph. It has been demonstrated that graph-based methods have high accuracy for retrieval by spatial similarity. However, naïvely representing all relationships on a complete graph obscures the structure of the tumour-anatomy relationships. We propose a new graph structure derived from complete graphs that structurally constrains the edges connected to tumour vertices based upon the spatial proximity of tumours and organs. This enables retrieval on the basis of tumour localisation. We also present a similarity matching algorithm that accounts for different feature sets for graph elements from different imaging modalities. Our method emphasises the relationships between a tumour and related organs, while still modelling patient-specific anatomical variations. Constraining tumours to related anatomical structures improves the discrimination potential of graphs, making it easier to retrieve similar images based on tumour location. We evaluated our retrieval methodology on a dataset of clinical PET-CT volumes. Our results showed that our method enabled the retrieval of multi-modality images using spatial features. Our graph-based retrieval algorithm achieved a higher precision than several other retrieval techniques: gray-level histograms as well as state-of-the-art methods such as visual words using the scale- invariant feature transform (SIFT) and relational matrices representing the spatial arrangements of objects.
authors.affiliation: Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia. Electronic address: ashnil.kumar@sydney.edu.au.
authors.forename: Ashnil
authors.initials: A
authors.lastname: Kumar
authors.affiliation: Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia. Electronic address: jinman.kim@sydney.edu.au.
authors.forename: Jinman
authors.initials: J
authors.lastname: Kim
authors.affiliation: Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia; Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia. Electronic address: lingfeng.wen@sydney.edu.au.
authors.forename: Lingfeng
authors.initials: L
authors.lastname: Wen
authors.affiliation: Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia; Sydney Medical School, University of Sydney, Sydney, Australia. Electronic address: michael.fulham@sydney.edu.au.
authors.forename: Michael
authors.initials: M
authors.lastname: Fulham
authors.affiliation: Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Sydney, Australia; Med-X Research Institute, Shanghai Jiao Tong University, China. Electronic address: dagan.feng@sydney.edu.au.
authors.forename: Dagan
authors.initials: D
authors.lastname: Feng
mesh_terms: D000465:Algorithms; D019788:Fluorodeoxyglucose F18; D006801:Humans; D008175:Lung Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology; D064847:Multimodal Imaging; D049268:Positron-Emission Tomography; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D019275:Radiopharmaceuticals; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed
chemical_list: D019275:Radiopharmaceuticals; D019788:Fluorodeoxyglucose F18
keywords: Content-based image retrieval; Graph similarity; Multi-modality; PET-CT