title: Ontology-guided organ detection to retrieve web images of disease manifestation: towards the construction of a consumer-based health image library.
abstract: BACKGROUND
  Visual information is a crucial aspect of medical knowledge. Building a comprehensive medical image base, in the spirit of the Unified Medical Language System (UMLS), would greatly benefit patient education and self-care. However, collection and annotation of such a large-scale image base is challenging.
  
  
  OBJECTIVE
  To combine visual object detection techniques with medical ontology to automatically mine web photos and retrieve a large number of disease manifestation images with minimal manual labeling effort.
  
  
  METHODS
  As a proof of concept, we first learnt five organ detectors on three detection scales for eyes, ears, lips, hands, and feet. Given a disease, we used information from the UMLS to select affected body parts, ran the pretrained organ detectors on web images, and combined the detection outputs to retrieve disease images.
  
  
  RESULTS
  Compared with a supervised image retrieval approach that requires training images for every disease, our ontology-guided approach exploits shared visual information of body parts across diseases. In retrieving 2220 web images of 32 diseases, we reduced manual labeling effort to 15.6% while improving the average precision by 3.9% from 77.7% to 81.6%. For 40.6% of the diseases, we improved the precision by 10%.
  
  
  CONCLUSIONS
  The results confirm the concept that the web is a feasible source for automatic disease image retrieval for health image database construction. Our approach requires a small amount of manual effort to collect complex disease images, and to annotate them by standard medical ontology terms.
authors.affiliation: Department of Electrical Engineering and Computer Science, Case Western Reserve University School of Engineering, Cleveland, Ohio, USA.
authors.forename: Yang
authors.initials: Y
authors.lastname: Chen
authors.forename: Xiaofeng
authors.initials: X
authors.lastname: Ren
authors.forename: Guo-Qiang
authors.initials: GQ
authors.lastname: Zhang
authors.forename: Rong
authors.initials: R
authors.lastname: Xu
mesh_terms: D054626:Consumer Health Information; D004194:Disease; D005528:Foot; D006225:Hand; D006257:Head; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D020407:Internet; D010781:Photography; D017432:Unified Medical Language System
keywords: Image Retrieval; Object Detection; Ontology