title: Improved medical image modality classification using a combination of visual and textual features.
abstract: In this paper, we present the approach that we applied to the medical modality classification tasks at the ImageCLEF evaluation forum. More specifically, we used the modality classification databases from the ImageCLEF competitions in 2011, 2012 and 2013, described by four visual and one textual types of features, and combinations thereof. We used local binary patterns, color and edge directivity descriptors, fuzzy color and texture histogram and scale-invariant feature transform (and its variant opponentSIFT) as visual features and the standard bag-of-words textual representation coupled with TF-IDF weighting. The results from the extensive experimental evaluation identify the SIFT and opponentSIFT features as the best performing features for modality classification. Next, the low-level fusion of the visual features improves the predictive performance of the classifiers. This is because the different features are able to capture different aspects of an image, their combination offering a more complete representation of the visual content in an image. Moreover, adding textual features further increases the predictive performance. Finally, the results obtained with our approach are the best results reported on these databases so far.
authors.affiliation: Faculty of Computer Science and Engineering, University Ss. Cyril and Methodius, Skopje, Macedonia. Electronic address: Ivica.Dimitrovski@finki.ukim.mk.
authors.forename: Ivica
authors.initials: I
authors.lastname: Dimitrovski
authors.affiliation: Department of Knowledge Technologies, Jožef Stefan Institute, Ljubljana, Slovenia. Electronic address: Dragi.Kocev@ijs.si.
authors.forename: Dragi
authors.initials: D
authors.lastname: Kocev
authors.affiliation: Faculty of Computer Science and Engineering, University Ss. Cyril and Methodius, Skopje, Macedonia. Electronic address: Ivan.Kitanovski@finki.ukim.mk.
authors.forename: Ivan
authors.initials: I
authors.lastname: Kitanovski
authors.affiliation: Faculty of Computer Science and Engineering, University Ss. Cyril and Methodius, Skopje, Macedonia. Electronic address: Suzana.Loskovska@finki.ukim.mk.
authors.forename: Suzana
authors.initials: S
authors.lastname: Loskovska
authors.affiliation: Department of Knowledge Technologies, Jožef Stefan Institute, Ljubljana, Slovenia. Electronic address: Saso.Dzeroski@ijs.si.
authors.forename: Sašo
authors.initials: S
authors.lastname: Džeroski
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D004282:Documentation / Q000379:methods*; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods*; D011873:Radiology Information Systems / Q000458:organization & administration*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D009626:Terminology as Topic; D014584:User-Computer Interface
keywords: Feature fusion; Image modality classification; Visual image descriptors