title: Mutational pattern of a sample from a critical branching population.
abstract: We study a universal object for the genealogy of a sample in populations with mutations: the critical birth-death process with Poissonian mutations, conditioned on its population size at a fixed time horizon. We show how this process arises as the law of the genealogy of a sample in a large class of nearly critical branching populations with rare mutations at birth, namely populations converging, in a large population asymptotic, towards the continuum random tree. We extend this model to populations with random foundation times, with (potentially improper) prior distributions [Formula: see text], [Formula: see text], including the so-called uniform ([Formula: see text]) and log-uniform ([Formula: see text]) priors. We first investigate the mutational patterns arising from these models, by studying the site frequency spectrum of a sample with fixed size, i.e. the number of mutations carried by k individuals in the sample. Explicit formulae for the expected frequency spectrum of a sample are provided, in the cases of a fixed foundation time, and of a uniform and log-uniform prior on the foundation time. Second, we establish the convergence in distribution, for large sample sizes, of the (suitably renormalized) tree spanned by the sample with prior [Formula: see text] on the time of origin. We finally prove that the limiting genealogies with different priors can all be embedded in the same realization of a given Poisson point measure.
authors.affiliation: Laboratoire de Probabilités et Modèles Aléatoires, UMR 7599 CNRS and UPMC Univ Paris 06, Paris, France.|Centre Interdisciplinaire de Recherche en Biologie, UMR 7241 CNRS and Collège de France, Paris, France.
authors.forename: Cécile
authors.initials: C
authors.lastname: Delaporte
authors.affiliation: Centre Interdisciplinaire de Recherche en Biologie, UMR 7241 CNRS and Collège de France, Paris, France.|Évolution Paris-Seine, UMR 7138 CNRS and UPMC Univ Paris 06, Paris, France.|Atelier de Bioinformatique, MNHN, Paris, France.
authors.forename: Guillaume
authors.initials: G
authors.lastname: Achaz
authors.affiliation: Laboratoire de Probabilités et Modèles Aléatoires, UMR 7599 CNRS and UPMC Univ Paris 06, Paris, France. amaury.lambert@upmc.fr.|Centre Interdisciplinaire de Recherche en Biologie, UMR 7241 CNRS and Collège de France, Paris, France. amaury.lambert@upmc.fr.
authors.forename: Amaury
authors.initials: A
authors.lastname: Lambert
mesh_terms: D005828:Genetics, Population; D006801:Humans; D008957:Models, Genetic; D009154:Mutation; D010802:Phylogeny
keywords: Coalescent point process; Critical birth-death process; Infinite-site model; Invariance principle; Poisson point measure; Sampling; Site frequency spectrum