title: The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience.
abstract: We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that buildconnectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.
authors.affiliation: Department of Computer Science and the Institute for Data Intensive Engineering and Science, Johns Hopkins University.
authors.forename: Randal
authors.initials: R
authors.lastname: Burns
authors.affiliation: Johns Hopkins University Applied Physics Laboratory.
authors.forename: William Gray
authors.initials: WG
authors.lastname: Roncal
authors.affiliation: Department of Statistical Science and Mathematics and the Institute for Brain Science, Duke University.
authors.forename: Dean
authors.initials: D
authors.lastname: Kleissas
authors.affiliation: Janelia Farm Research Campus, Howard Hughes Medical Institute.
authors.forename: Kunal
authors.initials: K
authors.lastname: Lillaney
authors.affiliation: Department of Molecular and Cellular Biology, Harvard University.
authors.forename: Priya
authors.initials: P
authors.lastname: Manavalan
authors.affiliation: Department of Computational Neuroscience, Massachusetts Institute of Technology.
authors.forename: Eric
authors.initials: E
authors.lastname: Perlman
authors.affiliation: Allen Institute for Brain Science.
authors.forename: Daniel R
authors.initials: DR
authors.lastname: Berger
authors.affiliation: Department of Physics and Astronomy and the Institute for Data Intensive Engineering and Science, Johns Hopkins University.
authors.forename: Davi D
authors.initials: DD
authors.lastname: Bock
authors.affiliation: Department of Bioengineering, Stanford University.
authors.forename: Kwanghun
authors.initials: K
authors.lastname: Chung
authors.affiliation: Department of Molecular and Cellular Physiology, Stanford University.
authors.forename: Logan
authors.initials: L
authors.lastname: Grosenick
authors.forename: Narayanan
authors.initials: N
authors.lastname: Kasthuri
authors.forename: Nicholas C
authors.initials: NC
authors.lastname: Weiler
authors.forename: Karl
authors.initials: K
authors.lastname: Deisseroth
authors.forename: Michael
authors.initials: M
authors.lastname: Kazhdan
authors.forename: Jeff
authors.initials: J
authors.lastname: Lichtman
authors.forename: R Clay
authors.initials: RC
authors.lastname: Reid
authors.forename: Stephen J
authors.initials: SJ
authors.lastname: Smith
authors.forename: Alexander S
authors.initials: AS
authors.lastname: Szalay
authors.forename: Joshua T
authors.initials: JT
authors.lastname: Vogelstein
authors.forename: R Jacob
authors.initials: RJ
authors.lastname: Vogelstein
keywords: Connectomics; Data-intensive computing