title: Crowdsourcing the creation of image segmentation algorithms for connectomics.
abstract: To stimulate progress in automating the reconstruction of neural circuits, we organized the first international challenge on 2D segmentation of electron microscopic (EM) images of the brain. Participants submitted boundary maps predicted for a test set of images, and were scored based on their agreement with a consensus of human expert annotations. The winning team had no prior experience with EM images, and employed a convolutional network. This "deep learning" approach has since become accepted as a standard for segmentation of EM images. The challenge has continued to accept submissions, and the best so far has resulted from cooperation between two teams. The challenge has probably saturated, as algorithms cannot progress beyond limits set by ambiguities inherent in 2D scoring and the size of the test dataset. Retrospective evaluation of the challenge scoring system reveals that it was not sufficiently robust to variations in the widths of neurite borders. We propose a solution to this problem, which should be useful for a future 3D segmentation challenge.
authors.affiliation: UMR1318 French National Institute for Agricultural Research-AgroParisTech, French National Institute for Agricultural Research Centre de Versailles-Grignon, Institut Jean-Pierre Bourgin Versailles, France.
authors.forename: Ignacio
authors.initials: I
authors.lastname: Arganda-Carreras
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus Ashburn, VA, USA.
authors.forename: Srinivas C
authors.initials: SC
authors.lastname: Turaga
authors.affiliation: Center for Brain Science, Harvard University Cambridge, MA, USA.
authors.forename: Daniel R
authors.initials: DR
authors.lastname: Berger
authors.affiliation: Swiss AI Lab IDSIA (Dalle Molle Institute for Artificial Intelligence) Universitá Della Svizzera Italiana, Scuola Universitaria Professionale Della Svizzera Italiana Lugano, Switzerland.
authors.forename: Dan
authors.initials: D
authors.lastname: Cireşan
authors.affiliation: Swiss AI Lab IDSIA (Dalle Molle Institute for Artificial Intelligence) Universitá Della Svizzera Italiana, Scuola Universitaria Professionale Della Svizzera Italiana Lugano, Switzerland.
authors.forename: Alessandro
authors.initials: A
authors.lastname: Giusti
authors.affiliation: Swiss AI Lab IDSIA (Dalle Molle Institute for Artificial Intelligence) Universitá Della Svizzera Italiana, Scuola Universitaria Professionale Della Svizzera Italiana Lugano, Switzerland.
authors.forename: Luca M
authors.initials: LM
authors.lastname: Gambardella
authors.affiliation: Swiss AI Lab IDSIA (Dalle Molle Institute for Artificial Intelligence) Universitá Della Svizzera Italiana, Scuola Universitaria Professionale Della Svizzera Italiana Lugano, Switzerland.
authors.forename: Jürgen
authors.initials: J
authors.lastname: Schmidhuber
authors.affiliation: Department of Computer Science, ETH Zurich Zurich, Switzerland.
authors.forename: Dmitry
authors.initials: D
authors.lastname: Laptev
authors.affiliation: Department of Computer Science, ETH Zurich Zurich, Switzerland.
authors.forename: Sarvesh
authors.initials: S
authors.lastname: Dwivedi
authors.affiliation: Department of Computer Science, ETH Zurich Zurich, Switzerland.
authors.forename: Joachim M
authors.initials: JM
authors.lastname: Buhmann
authors.affiliation: Scientific Computing and Imaging Institute, University of Utah Salt Lake City, UT, USA.
authors.forename: Ting
authors.initials: T
authors.lastname: Liu
authors.affiliation: Scientific Computing and Imaging Institute, University of Utah Salt Lake City, UT, USA.
authors.forename: Mojtaba
authors.initials: M
authors.lastname: Seyedhosseini
authors.affiliation: Scientific Computing and Imaging Institute, University of Utah Salt Lake City, UT, USA.
authors.forename: Tolga
authors.initials: T
authors.lastname: Tasdizen
authors.affiliation: Imaging Platform, Broad Institute Cambridge, MA, USA.
authors.forename: Lee
authors.initials: L
authors.lastname: Kamentsky
authors.affiliation: Department of Telecommunications, Faculty of Electrical Engineering and Communication, Brno University of Technology Brno, Czech Republic.
authors.forename: Radim
authors.initials: R
authors.lastname: Burget
authors.affiliation: Department of Telecommunications, Faculty of Electrical Engineering and Communication, Brno University of Technology Brno, Czech Republic.
authors.forename: Vaclav
authors.initials: V
authors.lastname: Uher
authors.affiliation: School of Engineering and Information Technology, University of New South Wales Canberra, ACT, Australia.
authors.forename: Xiao
authors.initials: X
authors.lastname: Tan
authors.affiliation: Digital Productivity Flagship, Commonwealth Scientific and Industrial Research Organisation North Ryde, NSW, Australia.
authors.forename: Changming
authors.initials: C
authors.lastname: Sun
authors.affiliation: Department of Biomedical Engineering, The Institute of Technology, Linkoping University Linkoping, Sweden.
authors.forename: Tuan D
authors.initials: TD
authors.lastname: Pham
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus Ashburn, VA, USA.
authors.forename: Erhan
authors.initials: E
authors.lastname: Bas
authors.affiliation: Computer Science Department, Rutgers University New Brunswick, NJ, USA.
authors.forename: Mustafa G
authors.initials: MG
authors.lastname: Uzunbas
authors.affiliation: Howard Hughes Medical Institute, Janelia Research Campus Ashburn, VA, USA.
authors.forename: Albert
authors.initials: A
authors.lastname: Cardona
authors.affiliation: Laboratory for Optical and Computational Instrumentation, University of Wisconsin-Madison Madison, WI, USA.
authors.forename: Johannes
authors.initials: J
authors.lastname: Schindelin
authors.affiliation: Princeton Neuroscience Institute and Computer Science Department, Princeton University Princeton, NJ, USA.
authors.forename: H Sebastian
authors.initials: HS
authors.lastname: Seung
keywords: connectomics; electron microscopy; image segmentation; machine learning; reconstruction