title: New Python-based methods for data processing.
abstract: Current pixel-array detectors produce diffraction images at extreme data rates (of up to 2 TB h(-1)) that make severe demands on computational resources. New multiprocessing frameworks are required to achieve rapid data analysis, as it is important to be able to inspect the data quickly in order to guide the experiment in real time. By utilizing readily available web-serving tools that interact with the Python scripting language, it was possible to implement a high-throughput Bragg-spot analyzer (cctbx.spotfinder) that is presently in use at numerous synchrotron-radiation beamlines. Similarly, Python interoperability enabled the production of a new data-reduction package (cctbx.xfel) for serial femtosecond crystallography experiments at the Linac Coherent Light Source (LCLS). Future data-reduction efforts will need to focus on specialized problems such as the treatment of diffraction spots on interleaved lattices arising from multi-crystal specimens. In these challenging cases, accurate modeling of close-lying Bragg spots could benefit from the high-performance computing capabilities of graphics-processing units.
authors.affiliation: Physical Biosciences Division, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA. nksauter@lbl.gov
authors.forename: Nicholas K
authors.initials: NK
authors.lastname: Sauter
authors.forename: Johan
authors.initials: J
authors.lastname: Hattne
authors.forename: Ralf W
authors.initials: RW
authors.lastname: Grosse-Kunstleve
authors.forename: Nathaniel
authors.initials: N
authors.lastname: Echols
mesh_terms: D000465:Algorithms; D018360:Crystallography, X-Ray; D003627:Data Interpretation, Statistical; D001330:Electronic Data Processing / Q000379:methods*; D004583:Electrons; D006801:Humans; D007834:Lasers; D009113:Muramidase / Q000737:chemistry*; D012984:Software; D017356:Synchrotrons / Q000295:instrumentation*
chemical_list: D009113:Muramidase
keywords: cctbx; data processing; multiprocessing; reusable code