title: Constructing benchmark databases and protocols for medical image analysis: diabetic retinopathy.
abstract: We address the performance evaluation practices for developing medical image analysis methods, in particular, how to establish and share databases of medical images with verified ground truth and solid evaluation protocols. Such databases support the development of better algorithms, execution of profound method comparisons, and, consequently, technology transfer from research laboratories to clinical practice. For this purpose, we propose a framework consisting of reusable methods and tools for the laborious task of constructing a benchmark database. We provide a software tool for medical image annotation helping to collect class label, spatial span, and expert's confidence on lesions and a method to appropriately combine the manual segmentations from multiple experts. The tool and all necessary functionality for method evaluation are provided as public software packages. As a case study, we utilized the framework and tools to establish the DiaRetDB1 V2.1 database for benchmarking diabetic retinopathy detection algorithms. The database contains a set of retinal images, ground truth based on information from multiple experts, and a baseline algorithm for the detection of retinopathy lesions.
authors: Machine Vision and Pattern Recognition Laboratory, Department of Mathematics and Physics, Lappeenranta University of Technology, Skinnarilankatu 34, Lappeenranta, Finland.
authors: Tomi
authors: T
authors: Kauppi
authors: Joni-Kristian
authors: JK
authors: Kämäräinen
authors: Lasse
authors: L
authors: Lensu
authors: Valentina
authors: V
authors: Kalesnykiene
authors: Iiris
authors: I
authors: Sorri
authors: Hannu
authors: H
authors: Uusitalo
authors: Heikki
authors: H
authors: Kälviäinen
mesh_terms: D000465:Algorithms; D016208:Databases, Factual; D003930:Diabetic Retinopathy / Q000175:diagnosis* / Q000473:pathology*; D005103:Expert Systems; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D012984:Software
references: IEEE Trans Med Imaging. 2000 Mar;19(3):203-10
references: 10875704
references: IEEE Trans Med Imaging. 2004 Jul;23(7):903-21
references: 15250643
references: IEEE Trans Med Imaging. 2004 Apr;23(4):501-9
references: 15084075
references: IEEE Trans Med Imaging. 2010 Jan;29(1):185-95
references: 19822469
references: J Biomed Inform. 2005 Oct;38(5):404-15
references: 16198999
references: Annu Int Conf IEEE Eng Med Biol Soc. 2008;2008:2262-5
references: 19163150
references: Med Image Anal. 2006 Aug;10(4):578-97
references: 16861030
references: IEEE Trans Med Imaging. 2003 Aug;22(8):951-8
references: 12906249