title: Validating retinal fundus image analysis algorithms: issues and a proposal.
abstract: This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.
authors.affiliation: VAMPIRE project, School of Computing, University of Dundee, Dundee, United Kingdom. manueltrucco@computing.dundee.ac.uk
authors.forename: Emanuele
authors.initials: E
authors.lastname: Trucco
authors.forename: Alfredo
authors.initials: A
authors.lastname: Ruggeri
authors.forename: Thomas
authors.initials: T
authors.lastname: Karnowski
authors.forename: Luca
authors.initials: L
authors.lastname: Giancardo
authors.forename: Edward
authors.initials: E
authors.lastname: Chaum
authors.forename: Jean Pierre
authors.initials: JP
authors.lastname: Hubschman
authors.forename: Bashir
authors.initials: B
authors.lastname: Al-Diri
authors.forename: Carol Y
authors.initials: CY
authors.lastname: Cheung
authors.forename: Damon
authors.initials: D
authors.lastname: Wong
authors.forename: Michael
authors.initials: M
authors.lastname: Abràmoff
authors.forename: Gilbert
authors.initials: G
authors.lastname: Lim
authors.forename: Dinesh
authors.initials: D
authors.lastname: Kumar
authors.forename: Philippe
authors.initials: P
authors.lastname: Burlina
authors.forename: Neil M
authors.initials: NM
authors.lastname: Bressler
authors.forename: Herbert F
authors.initials: HF
authors.lastname: Jelinek
authors.forename: Fabrice
authors.initials: F
authors.lastname: Meriaudeau
authors.forename: Gwénolé
authors.initials: G
authors.lastname: Quellec
authors.forename: Tom
authors.initials: T
authors.lastname: Macgillivray
authors.forename: Bal
authors.initials: B
authors.lastname: Dhillon
mesh_terms: D000465:Algorithms; D005654:Fundus Oculi; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000592:standards*; D009887:Ophthalmoscopy / Q000592:standards*; D012015:Reference Standards; D015203:Reproducibility of Results; D012164:Retinal Diseases / Q000473:pathology*; D012984:Software / Q000592:standards
keywords: fundus image analysis; reference standards; validation