title: SMaSH: a benchmarking toolkit for human genome variant calling.
abstract: BACKGROUND
  Computational methods are essential to extract actionable information from raw sequencing data, and to thus fulfill the promise of next-generation sequencing technology. Unfortunately, computational tools developed to call variants from human sequencing data disagree on many of their predictions, and current methods to evaluate accuracy and computational performance are ad hoc and incomplete. Agreement on benchmarking variant calling methods would stimulate development of genomic processing tools and facilitate communication among researchers.
  
  
  RESULTS
  We propose SMaSH, a benchmarking methodology for evaluating germline variant calling algorithms. We generate synthetic datasets, organize and interpret a wide range of existing benchmarking data for real genomes and propose a set of accuracy and computational performance metrics for evaluating variant calling methods on these benchmarking data. Moreover, we illustrate the utility of SMaSH to evaluate the performance of some leading single-nucleotide polymorphism, indel and structural variant calling algorithms.
  
  
  METHODS
  We provide free and open access online to the SMaSH tool kit, along with detailed documentation, at smash.cs.berkeley.edu
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Ameet
authors.initials: A
authors.lastname: Talwalkar
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Jesse
authors.initials: J
authors.lastname: Liptrap
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Julie
authors.initials: J
authors.lastname: Newcomb
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Christopher
authors.initials: C
authors.lastname: Hartl
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Terhorst
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Kristal
authors.initials: K
authors.lastname: Curtis
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Ma'ayan
authors.initials: M
authors.lastname: Bresler
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Yun S
authors.initials: YS
authors.lastname: Song
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: Michael I
authors.initials: MI
authors.lastname: Jordan
authors.affiliation: Department of Electrical Engineering and Computer Science, UC Berkeley, Berkeley, CA 94720, USA, The Broad Institute of Harvard and MIT, Cambridge, MA 02142, USA and Department of Statistics, UC Berkeley, Berkeley, CA 94720, USA.
authors.forename: David
authors.initials: D
authors.lastname: Patterson
mesh_terms: D000465:Algorithms; D019295:Computational Biology / Q000379:methods*; D003627:Data Interpretation, Statistical; D030541:Databases, Genetic; D015894:Genome, Human; D023281:Genomics / Q000379:methods*; D059014:High-Throughput Nucleotide Sequencing; D006801:Humans; D054643:INDEL Mutation; D020641:Polymorphism, Single Nucleotide; D012984:Software