title: Subsampled open-reference clustering creates consistent, comprehensive OTU definitions and scales to billions of sequences.
abstract: We present a performance-optimized algorithm, subsampled open-reference OTU picking, for assigning marker gene (e.g., 16S rRNA) sequences generated on next-generation sequencing platforms to operational taxonomic units (OTUs) for microbial community analysis. This algorithm provides benefits over de novo OTU picking (clustering can be performed largely in parallel, reducing runtime) and closed-reference OTU picking (all reads are clustered, not only those that match a reference database sequence with high similarity). Because more of our algorithm can be run in parallel relative to "classic" open-reference OTU picking, it makes open-reference OTU picking tractable on massive amplicon sequence data sets (though on smaller data sets, "classic" open-reference OTU clustering is often faster). We illustrate that here by applying it to the first 15,000 samples sequenced for the Earth Microbiome Project (1.3 billion V4 16S rRNA amplicons). To the best of our knowledge, this is the largest OTU picking run ever performed, and we estimate that our new algorithm runs in less than 1/5 the time than would be required of "classic" open reference OTU picking. We show that subsampled open-reference OTU picking yields results that are highly correlated with those generated by "classic" open-reference OTU picking through comparisons on three well-studied datasets. An implementation of this algorithm is provided in the popular QIIME software package, which uses uclust for read clustering. All analyses were performed using QIIME's uclust wrappers, though we provide details (aided by the open-source code in our GitHub repository) that will allow implementation of subsampled open-reference OTU picking independently of QIIME (e.g., in a compiled programming language, where runtimes should be further reduced). Our analyses should generalize to other implementations of these OTU picking algorithms. Finally, we present a comparison of parameter settings in QIIME's OTU picking workflows and make recommendations on settings for these free parameters to optimize runtime without reducing the quality of the results. These optimized parameters can vastly decrease the runtime of uclust-based OTU picking in QIIME.
authors.affiliation: Center for Microbial Genetics and Genomics, Northern Arizona University , Flagstaff, AZ , USA ; Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai , New York, NY , USA.
authors.forename: Jai Ram
authors.initials: JR
authors.lastname: Rideout
authors.affiliation: State Key Laboratory of Organ Failure Prevention, and Department of Environmental Health, School of Public Health and Tropical Medicine, Southern Medical University , Guangzhou, Guangdong , China.
authors.forename: Yan
authors.initials: Y
authors.lastname: He
authors.affiliation: Department of Computer Science, University of Colorado Boulder , Boulder, CO , USA.
authors.forename: Jose A
authors.initials: JA
authors.lastname: Navas-Molina
authors.affiliation: Department of Molecular, Cellular, and Developmental Biology, University of Colorado at Boulder , Boulder, CO , USA.
authors.forename: William A
authors.initials: WA
authors.lastname: Walters
authors.affiliation: Department of Chemistry and Biochemistry, University of Colorado at Boulder , Boulder, CO , USA.
authors.forename: Luke K
authors.initials: LK
authors.lastname: Ursell
authors.affiliation: Graduate Program in Biophysical Sciences, University of Chicago , Chicago, IL , USA ; Institute for Genomics and Systems Biology, Argonne National Laboratory , Lemont, IL , USA.
authors.forename: Sean M
authors.initials: SM
authors.lastname: Gibbons
authors.affiliation: Department of Biological Sciences, Northern Arizona University , AZ , USA.
authors.forename: John
authors.initials: J
authors.lastname: Chase
authors.affiliation: Department of Computer Science, University of Colorado Boulder , Boulder, CO , USA ; BioFrontiers Institute, University of Colorado at Boulder , Boulder, CO , USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: McDonald
authors.affiliation: BioFrontiers Institute, University of Colorado at Boulder , Boulder, CO , USA.
authors.forename: Antonio
authors.initials: A
authors.lastname: Gonzalez
authors.affiliation: Department of Computer Science, University of Colorado Boulder , Boulder, CO , USA ; BioFrontiers Institute, University of Colorado at Boulder , Boulder, CO , USA.
authors.forename: Adam
authors.initials: A
authors.lastname: Robbins-Pianka
authors.affiliation: Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai , New York, NY , USA.
authors.forename: Jose C
authors.initials: JC
authors.lastname: Clemente
authors.affiliation: Institute for Genomics and Systems Biology, Argonne National Laboratory , Lemont, IL , USA ; Department of Ecology and Evolution, University of Chicago , Chicago, IL , USA.
authors.forename: Jack A
authors.initials: JA
authors.lastname: Gilbert
authors.affiliation: Department of Pathology and Laboratory Science, Warren Alpert Medical School, Brown University , Providence, RI , USA.
authors.forename: Susan M
authors.initials: SM
authors.lastname: Huse
authors.affiliation: State Key Laboratory of Organ Failure Prevention, and Department of Environmental Health, School of Public Health and Tropical Medicine, Southern Medical University , Guangzhou, Guangdong , China.
authors.forename: Hong-Wei
authors.initials: HW
authors.lastname: Zhou
authors.affiliation: BioFrontiers Institute, University of Colorado at Boulder , Boulder, CO , USA ; Howard Hughes Medical Institute , Boulder, CO , USA.
authors.forename: Rob
authors.initials: R
authors.lastname: Knight
authors.affiliation: Center for Microbial Genetics and Genomics, Northern Arizona University , Flagstaff, AZ , USA ; Department of Biological Sciences, Northern Arizona University , AZ , USA.
authors.forename: J Gregory
authors.initials: JG
authors.lastname: Caporaso
keywords: Bioinformatics; Microbial ecology; Microbiome; OTU picking; Qiime