title: A novel benchmark model for intelligent annotation of spectral-domain optical coherence tomography scans using the example of cyst annotation.
abstract: OBJECTIVE
  The lack of benchmark data in computational ophthalmology contributes to the challenging task of applying disease assessment and evaluate performance of machine learning based methods on retinal spectral domain optical coherence tomography (SD-OCT) scans. Presented here is a general framework for constructing a benchmark dataset for retinal image processing tasks such as cyst, vessel, and subretinal fluid segmentation and as a result, a benchmark dataset for cyst segmentation has been developed.
  
  
  METHODS
  First, a dataset captured by different SD-OCT vendors with different numbers of scans and pathology qualities are selected. Then a robust and intelligent method is used to evaluate performance of readers, partitioning the dataset into subsets. Subsets are then assigned to complementary readers for annotation with respect to a novel confidence based annotation protocol. Finally, reader annotations are combined based on their performance to generate final annotations.
  
  
  RESULTS
  The generated benchmark dataset for cyst segmentation comprises 26 SD-OCT scans with differing cyst qualities, collected from 4 different SD-OCT vendors to cover a wide variety of data. The dataset is partitioned into three subsets which are annotated by complementary readers based on a confidence based annotation protocol. Experimental results show annotations of complementary readers are combined efficiently with respect to their performance, generating accurate annotations.
  
  
  CONCLUSIONS
  Our results facilitate the process of generating benchmark datasets. Moreover the generated benchmark data set for cyst segmentation can be used reliably to train and test machine learning based methods.
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Ehsan
authors.initials: E
authors.lastname: Shahrian Varnousfaderani
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Jing
authors.initials: J
authors.lastname: Wu
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Wolf-Dieter
authors.initials: WD
authors.lastname: Vogl
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Ana-Maria
authors.initials: AM
authors.lastname: Philip
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Alessio
authors.initials: A
authors.lastname: Montuoro
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Roland
authors.initials: R
authors.lastname: Leitner
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Christian
authors.initials: C
authors.lastname: Simader
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Sebastian M
authors.initials: SM
authors.lastname: Waldstein
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria. Electronic address: bianca.gerendas@meduniwien.ac.at.
authors.forename: Bianca S
authors.initials: BS
authors.lastname: Gerendas
authors.affiliation: Christian Doppler Laboratory for Ophthalmic Image Analysis, Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Ursula
authors.initials: U
authors.lastname: Schmidt-Erfurth
mesh_terms: D006801:Humans; D008962:Models, Theoretical; D041623:Tomography, Optical Coherence / Q000379:methods*
keywords: Benchmark dataset; Cyst segmentation; SD-OCT