title: Detailed simulation of cancer exome sequencing data reveals differences and common limitations of variant callers.
abstract: BACKGROUND
  Next-generation sequencing of matched tumor and normal biopsy pairs has become a technology of paramount importance for precision cancer treatment. Sequencing costs have dropped tremendously, allowing the sequencing of the whole exome of tumors for just a fraction of the total treatment costs. However, clinicians and scientists cannot take full advantage of the generated data because the accuracy of analysis pipelines is limited. This particularly concerns the reliable identification of subclonal mutations in a cancer tissue sample with very low frequencies, which may be clinically relevant.
  
  
  RESULTS
  Using simulations based on kidney tumor data, we compared the performance of nine state-of-the-art variant callers, namely deepSNV, GATK HaplotypeCaller, GATK UnifiedGenotyper, JointSNVMix2, MuTect, SAMtools, SiNVICT, SomaticSniper, and VarScan2. The comparison was done as a function of variant allele frequencies and coverage. Our analysis revealed that deepSNV and JointSNVMix2 perform very well, especially in the low-frequency range. We attributed false positive and false negative calls of the nine tools to specific error sources and assigned them to processing steps of the pipeline. All of these errors can be expected to occur in real data sets. We found that modifying certain steps of the pipeline or parameters of the tools can lead to substantial improvements in performance. Furthermore, a novel integration strategy that combines the ranks of the variants yielded the best performance. More precisely, the rank-combination of deepSNV, JointSNVMix2, MuTect, SiNVICT and VarScan2 reached a sensitivity of 78% when fixing the precision at 90%, and outperformed all individual tools, where the maximum sensitivity was 71% with the same precision.
  
  
  CONCLUSIONS
  The choice of well-performing tools for alignment and variant calling is crucial for the correct interpretation of exome sequencing data obtained from mixed samples, and common pipelines are suboptimal. We were able to relate observed substantial differences in performance to the underlying statistical models of the tools, and to pinpoint the error sources of false positive and false negative calls. These findings might inspire new software developments that improve exome sequencing pipelines and further the field of precision cancer treatment.
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland.|Swiss Institute of Bioinformatics, Mattenstr, Basel, 26, 4058, Switzerland.
authors.forename: Ariane L
authors.initials: AL
authors.lastname: Hofmann
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland.|Swiss Institute of Bioinformatics, Mattenstr, Basel, 26, 4058, Switzerland.
authors.forename: Jonas
authors.initials: J
authors.lastname: Behr
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland.|Swiss Institute of Bioinformatics, Mattenstr, Basel, 26, 4058, Switzerland.
authors.forename: Jochen
authors.initials: J
authors.lastname: Singer
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland.|Swiss Institute of Bioinformatics, Mattenstr, Basel, 26, 4058, Switzerland.
authors.forename: Jack
authors.initials: J
authors.lastname: Kuipers
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland.
authors.forename: Christian
authors.initials: C
authors.lastname: Beisel
authors.affiliation: Institute for Surgical Pathology, University Hospital Zurich, Schmelzbergstrasse 12, Zurich, 8091, Switzerland.
authors.forename: Peter
authors.initials: P
authors.lastname: Schraml
authors.affiliation: Institute for Surgical Pathology, University Hospital Zurich, Schmelzbergstrasse 12, Zurich, 8091, Switzerland.
authors.forename: Holger
authors.initials: H
authors.lastname: Moch
authors.affiliation: Department of Biosystems Science and Engineering, ETH Zurich, Mattenstr, Basel, 26, 4058, Switzerland. niko.beerenwinkel@bsse.ethz.ch.|Swiss Institute of Bioinformatics, Mattenstr, Basel, 26, 4058, Switzerland. niko.beerenwinkel@bsse.ethz.ch.
authors.forename: Niko
authors.initials: N
authors.lastname: Beerenwinkel
mesh_terms: D000465:Algorithms; D004273:DNA, Neoplasm / Q000737:chemistry / Q000378:metabolism; D059472:Exome / Q000235:genetics*; D023281:Genomics; D059014:High-Throughput Nucleotide Sequencing; D006801:Humans; D007680:Kidney Neoplasms / Q000235:genetics* / Q000473:pathology; D020641:Polymorphism, Single Nucleotide; D017422:Sequence Analysis, DNA
chemical_list: D004273:DNA, Neoplasm
keywords: Cancer genomics; Exome sequencing; SNV; Variant caller integration; Variant calling