title: Do we need annotation experts? A case study in celiac disease classification.
abstract: Inference of clinically-relevant findings from the visual appearance of images has become an essential part of processing pipelines for many problems in medical imaging. Typically, a sufficient amount labeled training data is assumed to be available, provided by domain experts. However, acquisition of this data is usually a time-consuming and expensive endeavor. In this work, we ask the question if, for certain problems, expert knowledge is actually required. In fact, we investigate the impact of letting non-expert volunteers annotate a database of endoscopy images which are then used to assess the absence/presence of celiac disease. Contrary to previous approaches, we are not interested in algorithms that can handle the label noise. Instead, we present compelling empirical evidence that label noise can be compensated by a sufficiently large corpus of training data, labeled by the non-experts.
authors.forename: Roland
authors.initials: R
authors.lastname: Kwitt
authors.forename: Sebastian
authors.initials: S
authors.lastname: Hegenbart
authors.forename: Nikhil
authors.initials: N
authors.lastname: Rasiwasia
authors.forename: Andreas
authors.initials: A
authors.lastname: Vécsei
authors.forename: Andreas
authors.initials: A
authors.lastname: Uhl
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D002446:Celiac Disease / Q000473:pathology*; D063045:Crowdsourcing / Q000379:methods*; D004724:Endoscopy / Q000379:methods*; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity