title: TED: A Tolerant Edit Distance for segmentation evaluation.
abstract: In this paper, we present a novel error measure to compare a computer-generated segmentation of images or volumes against ground truth. This measure, which we call Tolerant Edit Distance (TED), is motivated by two observations that we usually encounter in biomedical image processing: (1) Some errors, like small boundary shifts, are tolerable in practice. Which errors are tolerable is application dependent and should be explicitly expressible in the measure. (2) Non-tolerable errors have to be corrected manually. The effort needed to do so should be reflected by the error measure. Our measure is the minimal weighted sum of split and merge operations to apply to one segmentation such that it resembles another segmentation within specified tolerance bounds. This is in contrast to other commonly used measures like Rand index or variation of information, which integrate small, but tolerable, differences. Additionally, the TED provides intuitive numbers and allows the localization and classification of errors in images or volumes. We demonstrate the applicability of the TED on 3D segmentations of neurons in electron microscopy images where topological correctness is arguable more important than exact boundary locations. Furthermore, we show that the TED is not just limited to evaluation tasks. We use it as the loss function in a max-margin learning framework to find parameters of an automatic neuron segmentation algorithm. We show that training to minimize the TED, i.e., to minimize crucial errors, leads to higher segmentation accuracy compared to other learning methods.
authors.affiliation: Institut de Robòtica i Informàtica Industrial, UPC/CSIC, Barcelona, Spain; Janelia Research Campus, VA, Ashburn, United States; Institute of Neuroinformatics, UZH/ETH, Zurich, Switzerland. Electronic address: jfunke@iri.upc.edu.
authors.forename: Jan
authors.initials: J
authors.lastname: Funke
authors.affiliation: Institute of Neuroinformatics, UZH/ETH, Zurich, Switzerland. Electronic address: klein@ini.uzh.ch.
authors.forename: Jonas
authors.initials: J
authors.lastname: Klein
authors.affiliation: Institut de Robòtica i Informàtica Industrial, UPC/CSIC, Barcelona, Spain. Electronic address: fmoreno@iri.upc.edu.
authors.forename: Francesc
authors.initials: F
authors.lastname: Moreno-Noguer
authors.affiliation: Janelia Research Campus, VA, Ashburn, United States. Electronic address: cardonaa@janelia.hhmi.org.
authors.forename: Albert
authors.initials: A
authors.lastname: Cardona
authors.affiliation: Institute of Neuroinformatics, UZH/ETH, Zurich, Switzerland. Electronic address: cook@ini.uzh.ch.
authors.forename: Matthew
authors.initials: M
authors.lastname: Cook
mesh_terms: D000704:Analysis of Variance; D000818:Animals; D002540:Cerebral Cortex / Q000033:anatomy & histology / Q000648:ultrastructure*; D004331:Drosophila melanogaster / Q000166:cytology / Q000648:ultrastructure; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods / Q000706:statistics & numerical data*; D000069550:Machine Learning; D051379:Mice; D008854:Microscopy, Electron / Q000706:statistics & numerical data*; D009474:Neurons / Q000166:cytology / Q000648:ultrastructure*; D010363:Pattern Recognition, Automated / Q000706:statistics & numerical data*
keywords: Computer vision; Electron microscopy; Evaluation; Learning; Neuron segmentation; Segmentation