title: On the importance of being structured: instantaneous coalescence rates and human evolution--lessons for ancestral population size inference?
abstract: Most species are structured and influenced by processes that either increased or reduced gene flow between populations. However, most population genetic inference methods assume panmixia and reconstruct a history characterized by population size changes. This is potentially problematic as population structure can generate spurious signals of population size change through time. Moreover, when the model assumed for demographic inference is misspecified, genomic data will likely increase the precision of misleading if not meaningless parameters. For instance, if data were generated under an n-island model (characterized by the number of islands and migrants exchanged) inference based on a model of population size change would produce precise estimates of a bottleneck that would be meaningless. In addition, archaeological or climatic events around the bottleneck's timing might provide a reasonable but potentially misleading scenario. In a context of model uncertainty (panmixia versus structure) genomic data may thus not necessarily lead to improved statistical inference. We consider two haploid genomes and develop a theory that explains why any demographic model with structure will necessarily be interpreted as a series of changes in population size by inference methods ignoring structure. We formalize a parameter, the inverse instantaneous coalescence rate, and show that it is equivalent to a population size only in panmictic models, and is mostly misleading for structured models. We argue that this issue affects all population genetics methods ignoring population structure which may thus infer population size changes that never took place. We apply our approach to human genomic data.
authors.affiliation: Université de Toulouse, Institut National des Sciences Appliquées, Institut de Mathématiques de Toulouse, Toulouse, France.
authors.forename: O
authors.initials: O
authors.lastname: Mazet
authors.affiliation: Université de Toulouse, Institut National des Sciences Appliquées, Institut de Mathématiques de Toulouse, Toulouse, France.
authors.forename: W
authors.initials: W
authors.lastname: Rodríguez
authors.affiliation: Université de Toulouse, Institut National des Sciences Appliquées, Institut de Mathématiques de Toulouse, Toulouse, France.
authors.forename: S
authors.initials: S
authors.lastname: Grusea
authors.affiliation: UMR7205 Institut de Systématique, Évolution et Biodiversité, École Pratique des Hautes Études & Muséum National d'Histoire Naturelle & CNRS and Université Pierre et Marie Curie, Paris, France.|UMR1313 Génétique Animale et Biologie Intégrative, Institut National de la Recherche Agronomique and AgroParisTech, Jouy-en-Josas, France.
authors.forename: S
authors.initials: S
authors.lastname: Boitard
authors.affiliation: CNRS, Université Paul Sabatier, ENFA, UMR 5174 EDB (Laboratoire Évolution and Diversité Biologique), Toulouse, France.|Université de Toulouse, UPS, EDB, Toulouse, France.|Instituto Gulbenkian de Ciência, Oeiras, Portugal.
authors.forename: L
authors.initials: L
authors.lastname: Chikhi
mesh_terms: D003710:Demography; D005828:Genetics, Population; D015894:Genome, Human; D023281:Genomics; D006238:Haploidy; D006801:Humans; D008957:Models, Genetic; D011156:Population Density