title: Population structure and coalescence in pedigrees: Comparisons to the structured coalescent and a framework for inference.
abstract: Contrary to what is often assumed in population genetics, independently segregating loci do not have completely independent ancestries, since all loci are inherited through a single, shared population pedigree. Previous work has shown that the non-independence between gene genealogies of independently segregating loci created by the population pedigree is weak in panmictic populations, and predictions made from standard coalescent theory are accurate for populations that are at least moderately sized. Here, we investigate patterns of coalescence in pedigrees of structured populations. We find that the pedigree creates deviations away from the predictions of the structured coalescent that persist on a longer timescale than in the case of panmictic populations. Nevertheless, we find that the structured coalescent provides a reasonable approximation for the coalescent process in structured population pedigrees so long as migration events are moderately frequent and there are no migration events in the recent pedigree of the sample. When there are migration events in the recent sample pedigree, we find that distributions of coalescence in the sample can be modeled as a mixture of distributions from different initial sample configurations. We use this observation to motivate a maximum-likelihood approach for inferring migration rates and mutation rates jointly with features of the pedigree such as recent migrant ancestry and recent relatedness. Using simulation, we show that our inference framework accurately recovers long-term migration rates in the presence of recent migration events in the sample pedigree.
authors.affiliation: Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA, 02138, USA. Electronic address: pwilton@fas.harvard.edu.
authors.forename: Peter R
authors.initials: PR
authors.lastname: Wilton
authors.affiliation: Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA, 02138, USA; École des Mines de Paris, Mines ParisTech, Paris 75272, France.
authors.forename: Pierre
authors.initials: P
authors.lastname: Baduel
authors.affiliation: Department of Systems Biology, Harvard University, Cambridge, MA, 02138, USA; École des Mines de Paris, Mines ParisTech, Paris 75272, France.
authors.forename: Matthieu M
authors.initials: MM
authors.lastname: Landon
authors.affiliation: Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA, 02138, USA.
authors.forename: John
authors.initials: J
authors.lastname: Wakeley
mesh_terms: D005789:Genealogy and Heraldry; D005828:Genetics, Population / Q000379:methods*; D006801:Humans; D016013:Likelihood Functions; D008957:Models, Genetic; D010375:Pedigree
keywords: Coalescent theory; Demographic inference; Identity-by-descent; Pedigree