title: What is genetic differentiation, and how should we measure it--GST, D, neither or both?
abstract: Estimates of the fixation index, F(ST), have been used as measures of population differentiation for many decades. However, there have been persistent voices in the literature suggesting that these statistics do not measure true differentiation. In particular, the statistics Nei's G(ST) and Wier and Cockerham's θ have been criticized for being 'constrained' to not equal one in some situations that seem to represent maximal differentiation. Here, we address the issue of how to evaluate exactly how much information a particular statistic contains about the process of differentiation. This criterion can be used to counter most concerns about the performance of G(ST) (and related statistics), while also being reconciled with the insights of those who have proposed alternative measures of differentiation. In particular, the likelihood-based framework that we put forward can justify the use of G(ST) as an effective measure of differentiation, but also shows that in some situations G(ST) is insufficient on its own and needs supplementing by another measure such as Jost's D or Hedrick's G'(ST). This approach will become increasingly important in the future, as greater emphasis is placed on analysing large data sets.
authors.affiliation: Medical Research Council Centre for Outbreak Analysis and Modelling, Department of Infectious Disease Epidemiology, Imperial College London, London, W2 1PG, UK; School of Biological and Chemical Sciences, Queen Mary University of London, Mile End Road, London, E1 4NS, UK.
authors.forename: Robert
authors.initials: R
authors.lastname: Verity
authors.forename: Richard A
authors.initials: RA
authors.lastname: Nichols
mesh_terms: D005787:Gene Frequency; D040961:Genetic Drift; D005828:Genetics, Population / Q000379:methods*; D006579:Heterozygote; D016013:Likelihood Functions; D008957:Models, Genetic; D015233:Models, Statistical; D009154:Mutation
keywords: Hedrick's G′ST; Jost's D; differentiation; fixation index; information; likelihood