title: Uncovering the Horseshoe Effect in Microbial Analyses.
abstract: The horseshoe effect is a phenomenon that has long intrigued ecologists. The effect was commonly thought to be an artifact of dimensionality reduction, and multiple techniques were developed to unravel this phenomenon and simplify interpretation. Here, we provide evidence that horseshoes arise as a consequence of distance metrics that saturate-a familiar concept in other fields but new to microbial ecology. This saturation property loses information about community dissimilarity, simply because it cannot discriminate between samples that do not share any common features. The phenomenon illuminates niche differentiation in microbial communities and indicates species turnover along environmental gradients. Here we propose a rationale for the observed horseshoe effect from multiple dimensionality reduction techniques applied to simulations, soil samples, and samples from postmortem mice. An in-depth understanding of this phenomenon allows targeting of niche differentiation patterns from high-level ordination plots, which can guide conventional statistical tools to pinpoint microbial niches along environmental gradients.IMPORTANCEThe horseshoe effect is often considered an artifact of dimensionality reduction. We show that this is not true in the case for microbiome data and that, in fact, horseshoes can help analysts discover microbial niches across environments.
authors.affiliation: Department of Computer Science, University of California, San Diego, La Jolla, California, USA; Department of Pediatrics, University of California, San Diego, La Jolla, California, USA.
authors.forename: James T
authors.initials: JT
authors.lastname: Morton
authors.affiliation: Department of Mathematics, École Normale Supérieure de Lyon, Lyon, France.
authors.forename: Liam
authors.initials: L
authors.lastname: Toran
authors.affiliation: Genomic Medicine Group, J. Craig Venter Institute, La Jolla, California, USA.
authors.forename: Anna
authors.initials: A
authors.lastname: Edlund
authors.affiliation: Department of Animal Sciences, Colorado State University, Fort Collins, Colorado, USA.
authors.forename: Jessica L
authors.initials: JL
authors.lastname: Metcalf
authors.affiliation: Nestle Institute of Health Sciences, Lausanne, Switzerland.
authors.forename: Christian
authors.initials: C
authors.lastname: Lauber
authors.affiliation: Department of Computer Science, University of California, San Diego, La Jolla, California, USA; Department of Pediatrics, University of California, San Diego, La Jolla, California, USA.
authors.forename: Rob
authors.initials: R
authors.lastname: Knight
keywords: decomposition; horseshoe; microbial ecology; pH; soil