title: An Improved F(st) Estimator.
abstract: The fixation index F(st) plays a central role in ecological and evolutionary genetic studies. The estimators of Wright ([Formula: see text]), Weir and Cockerham ([Formula: see text]), and Hudson et al. ([Formula: see text]) are widely used to measure genetic differences among different populations, but all have limitations. We propose a minimum variance estimator [Formula: see text] using [Formula: see text] and [Formula: see text]. We tested [Formula: see text] in simulations and applied it to 120 unrelated East African individuals from Ethiopia and 11 subpopulations in HapMap 3 with 464,642 SNPs. Our simulation study showed that [Formula: see text] has smaller bias than [Formula: see text] for small sample sizes and smaller bias than [Formula: see text] for large sample sizes. Also, [Formula: see text] has smaller variance than [Formula: see text] for small Fst values and smaller variance than [Formula: see text] for large F(st) values. We demonstrated that approximately 30 subpopulations and 30 individuals per subpopulation are required in order to accurately estimate F(st).
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Guanjie
authors: G
authors: Chen
authors: Department of Biostatistics, Bioinformatics and Biomathematics, Georgetown University, Washington, DC, United States of America.
authors: Ao
authors: A
authors: Yuan
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Daniel
authors: D
authors: Shriner
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Fasil
authors: F
authors: Tekola-Ayele
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Jie
authors: J
authors: Zhou
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Amy R
authors: AR
authors: Bentley
authors: Suizhou Central Hospital, Suizhou, Hubei, China.
authors: Yanxun
authors: Y
authors: Zhou
authors: Suizhou Central Hospital, Suizhou, Hubei, China.
authors: Chuntao
authors: C
authors: Wang
authors: Brighton and Sussex Medical School, Falmer, Brighton, United Kingdom.
authors: Melanie J
authors: MJ
authors: Newport
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Adebowale
authors: A
authors: Adeyemo
authors: Center for Research on Genomics and Global Health, NHGRI, NIH, Bethesda, Maryland, United States of America.
authors: Charles N
authors: CN
authors: Rotimi
mesh_terms: D000465:Algorithms; D008962:Models, Theoretical