title: Adapting content-based image retrieval techniques for the semantic annotation of medical images.
abstract: The automatic annotation of medical images is a prerequisite for building comprehensive semantic archives that can be used to enhance evidence-based diagnosis, physician education, and biomedical research. Annotation also has important applications in the automatic generation of structured radiology reports. Much of the prior research work has focused on annotating images with properties such as the modality of the image, or the biological system or body region being imaged. However, many challenges remain for the annotation of high-level semantic content in medical images (e.g., presence of calcification, vessel obstruction, etc.) due to the difficulty in discovering relationships and associations between low-level image features and high-level semantic concepts. This difficulty is further compounded by the lack of labelled training data. In this paper, we present a method for the automatic semantic annotation of medical images that leverages techniques from content-based image retrieval (CBIR). CBIR is a well-established image search technology that uses quantifiable low-level image features to represent the high-level semantic content depicted in those images. Our method extends CBIR techniques to identify or retrieve a collection of labelled images that have similar low-level features and then uses this collection to determine the best high-level semantic annotations. We demonstrate our annotation method using retrieval via weighted nearest-neighbour retrieval and multi-class classification to show that our approach is viable regardless of the underlying retrieval strategy. We experimentally compared our method with several well-established baseline techniques (classification and regression) and showed that our method achieved the highest accuracy in the annotation of liver computed tomography (CT) images.
authors.affiliation: School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: ashnil.kumar@sydney.edu.au.
authors.forename: Ashnil
authors.initials: A
authors.lastname: Kumar
authors.affiliation: School of Electrical and Information Engineering, University of Sydney, Australia. Electronic address: sdye9175@uni.sydney.edu.au.
authors.forename: Shane
authors.initials: S
authors.lastname: Dyer
authors.affiliation: School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: jinman.kim@sydney.edu.au.
authors.forename: Jinman
authors.initials: J
authors.lastname: Kim
authors.affiliation: School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: changyang.li@sydney.edu.au.
authors.forename: Changyang
authors.initials: C
authors.lastname: Li
authors.affiliation: School of Electrical and Information Engineering, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: philip.leong@sydney.edu.au.
authors.forename: Philip H W
authors.initials: PH
authors.lastname: Leong
authors.affiliation: School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia; Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia; Sydney Medical School, University of Sydney, Australia. Electronic address: michael.fulham@sydney.edu.au.
authors.forename: Michael
authors.initials: M
authors.lastname: Fulham
authors.affiliation: School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia; Med-X Research Institute, Shanghai Jiao Tong University, China. Electronic address: dagan.feng@sydney.edu.au.
authors.forename: Dagan
authors.initials: D
authors.lastname: Feng
mesh_terms: D000465:Algorithms; D057225:Data Mining / Q000379:methods*; D003628:Database Management Systems; D004282:Documentation / Q000379:methods*; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000379:methods; D008099:Liver / Q000000981:diagnostic imaging*; D000069550:Machine Learning; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods; D011873:Radiology Information Systems / Q000458:organization & administration*; D015203:Reproducibility of Results; D012660:Semantics; D012680:Sensitivity and Specificity; D009626:Terminology as Topic; D014057:Tomography, X-Ray Computed / Q000379:methods*
keywords: Computed tomography; Content-based image retrieval; Image annotation; ImageCLEF; Liver