title: Cross-validation to select Bayesian hierarchical models in phylogenetics.
abstract: Recent developments in Bayesian phylogenetic models have increased the range of inferences that can be drawn from molecular sequence data. Accordingly, model selection has become an important component of phylogenetic analysis. Methods of model selection generally consider the likelihood of the data under the model in question. In the context of Bayesian phylogenetics, the most common approach involves estimating the marginal likelihood, which is typically done by integrating the likelihood across model parameters, weighted by the prior. Although this method is accurate, it is sensitive to the presence of improper priors. We explored an alternative approach based on cross-validation that is widely used in evolutionary analysis. This involves comparing models according to their predictive performance.
  
  
  
  We analysed simulated data and a range of viral and bacterial data sets using a cross-validation approach to compare a variety of molecular clock and demographic models. Our results show that cross-validation can be effective in distinguishing between strict- and relaxed-clock models and in identifying demographic models that allow growth in population size over time. In most of our empirical data analyses, the model selected using cross-validation was able to match that selected using marginal-likelihood estimation. The accuracy of cross-validation appears to improve with longer sequence data, particularly when distinguishing between relaxed-clock models.
  
  
  
  Cross-validation is a useful method for Bayesian phylogenetic model selection. This method can be readily implemented even when considering complex models where selecting an appropriate prior for all parameters may be difficult.
authors.affiliation: Marie Bashir Institute of Infectious Diseases and Biosecurity, Charles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, NSW, 2006, Australia. sebastian.duchene@sydney.edu.au.|School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia. sebastian.duchene@sydney.edu.au.
authors.forename: Sebastián
authors.initials: S
authors.lastname: Duchêne
authors.affiliation: School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: David A
authors.initials: DA
authors.lastname: Duchêne
authors.affiliation: Marie Bashir Institute of Infectious Diseases and Biosecurity, Charles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, NSW, 2006, Australia.|School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: Francesca
authors.initials: F
authors.lastname: Di Giallonardo
authors.affiliation: Marie Bashir Institute of Infectious Diseases and Biosecurity, Charles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, NSW, 2006, Australia.|School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: John-Sebastian
authors.initials: JS
authors.lastname: Eden
authors.affiliation: Marie Bashir Institute of Infectious Diseases and Biosecurity, Charles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, NSW, 2006, Australia.|School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: Jemma L
authors.initials: JL
authors.lastname: Geoghegan
authors.affiliation: Department of Biochemistry and Molecular Biology, Bio21 Molecular Science and Biotechnology Institute, The University of Melbourne, Melbourne, VIC, 3010, Australia.|Centre for Systems Genomics, The University of Melbourne, Melbourne, VIC, 3010, Australia.
authors.forename: Kathryn E
authors.initials: KE
authors.lastname: Holt
authors.affiliation: School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: Simon Y W
authors.initials: SY
authors.lastname: Ho
authors.affiliation: Marie Bashir Institute of Infectious Diseases and Biosecurity, Charles Perkins Centre, Sydney Medical School, University of Sydney, Sydney, NSW, 2006, Australia.|School of Life and Environmental Sciences, University of Sydney, Sydney, NSW, 2006, Australia.
authors.forename: Edward C
authors.initials: EC
authors.lastname: Holmes
mesh_terms: D001419:Bacteria / Q000235:genetics; D001499:Bayes Theorem; D005075:Biological Evolution; D003198:Computer Simulation; D019143:Evolution, Molecular; D016013:Likelihood Functions; D008957:Models, Genetic; D010802:Phylogeny; D015203:Reproducibility of Results; D014780:Viruses / Q000235:genetics
keywords: Bayesian phylogenetics; Cross-validation; Demographic models; Marginal likelihood; Model selection; Molecular clock