title: A random effects branch-site model for detecting episodic diversifying selection.
abstract: Adaptive evolution frequently occurs in episodic bursts, localized to a few sites in a gene, and to a small number of lineages in a phylogenetic tree. A popular class of "branch-site" evolutionary models provides a statistical framework to search for evidence of such episodic selection. For computational tractability, current branch-site models unrealistically assume that all branches in the tree can be partitioned a priori into two rigid classes--"foreground" branches that are allowed to undergo diversifying selective bursts and "background" branches that are negatively selected or neutral. We demonstrate that this assumption leads to unacceptably high rates of false positives or false negatives when the evolutionary process along background branches strongly deviates from modeling assumptions. To address this problem, we extend Felsenstein's pruning algorithm to allow efficient likelihood computations for models in which variation over branches (and not just sites) is described in the random effects likelihood framework. This enables us to model the process at every branch-site combination as a mixture of three Markov substitution models--our model treats the selective class of every branch at a particular site as an unobserved state that is chosen independently of that at any other branch. When benchmarked on a previously published set of simulated sequences, our method consistently matched or outperformed existing branch-site tests in terms of power and error rates. Using three empirical data sets, previously analyzed for episodic selection, we discuss how modeling assumptions can influence inference in practical situations.
authors.affiliation: Department of Medicine, University of California-San Diego, CA, USA. spond@ucsd.edu
authors.forename: Sergei L
authors.initials: SL
authors.lastname: Kosakovsky Pond
authors.forename: Ben
authors.initials: B
authors.lastname: Murrell
authors.forename: Mathieu
authors.initials: M
authors.lastname: Fourment
authors.forename: Simon D W
authors.initials: SD
authors.lastname: Frost
authors.forename: Wayne
authors.initials: W
authors.lastname: Delport
authors.forename: Konrad
authors.initials: K
authors.lastname: Scheffler
mesh_terms: D000465:Algorithms; D003062:Codon / Q000235:genetics; D019143:Evolution, Molecular; D016013:Likelihood Functions; D008390:Markov Chains; D008957:Models, Genetic; D010802:Phylogeny; D012641:Selection, Genetic
chemical_list: D003062:Codon