title: Trace: a high-throughput tomographic reconstruction engine for large-scale datasets.
abstract: BACKGROUND
  Modern synchrotron light sources and detectors produce data at such scale and complexity that large-scale computation is required to unleash their full power. One of the widely used imaging techniques that generates data at tens of gigabytes per second is computed tomography (CT). Although CT experiments result in rapid data generation, the analysis and reconstruction of the collected data may require hours or even days of computation time with a medium-sized workstation, which hinders the scientific progress that relies on the results of analysis.
  
  
  METHODS
  We present Trace, a data-intensive computing engine that we have developed to enable high-performance implementation of iterative tomographic reconstruction algorithms for parallel computers. Trace provides fine-grained reconstruction of tomography datasets using both (thread-level) shared memory and (process-level) distributed memory parallelization. Trace utilizes a special data structure called replicated reconstruction object to maximize application performance. We also present the optimizations that we apply to the replicated reconstruction objects and evaluate them using tomography datasets collected at the Advanced Photon Source.
  
  
  RESULTS
  Our experimental evaluations show that our optimizations and parallelization techniques can provide 158× speedup using 32 compute nodes (384 cores) over a single-core configuration and decrease the end-to-end processing time of a large sinogram (with 4501 × 1 × 22,400 dimensions) from 12.5 h to <5 min per iteration.
  
  
  CONCLUSIONS
  The proposed tomographic reconstruction engine can efficiently process large-scale tomographic data using many compute nodes and minimize reconstruction times.
authors.affiliation: Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.
authors.forename: Tekin
authors.identifier: 0000-0002-8428-5159
authors.initials: T
authors.lastname: Bicer
authors.affiliation: X-Ray Science Division, Advanced Photon Source, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.
authors.forename: Doğa
authors.initials: D
authors.lastname: Gürsoy
authors.affiliation: X-Ray Science Division, Advanced Photon Source, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.
authors.forename: Vincent De
authors.initials: V
authors.lastname: Andrade
authors.affiliation: Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.|Computation Institute, University of Chicago and Argonne National Laboratory, 5735 South Ellis Ave., Chicago, IL 60637 USA.
authors.forename: Rajkumar
authors.initials: R
authors.lastname: Kettimuthu
authors.affiliation: Argonne Leadership Computing Facility, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.
authors.forename: William
authors.initials: W
authors.lastname: Scullin
authors.affiliation: X-Ray Science Division, Advanced Photon Source, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.
authors.forename: Francesco De
authors.initials: F
authors.lastname: Carlo
authors.affiliation: Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Ave., Lemont, IL 60439 USA.|Computation Institute, University of Chicago and Argonne National Laboratory, 5735 South Ellis Ave., Chicago, IL 60637 USA.|Department of Computer Science, University of Chicago, 5801 South Ellis Ave., Chicago, IL 60637 USA.
authors.forename: Ian T
authors.initials: IT
authors.lastname: Foster
keywords: Big data; High-throughput; Reconstruction; Tomography