title: Evaluation of the potential excess of statistically significant findings in published genetic association studies: application to Alzheimer's disease.
abstract: The authors evaluated whether there is an excess of statistically significant results in studies of genetic associations with Alzheimer's disease reflecting either between-study heterogeneity or bias. Among published articles on genetic associations entered into the comprehensive AlzGene database (www.alzgene.org) through January 31, 2007, 1,348 studies included in 175 meta-analyses with 3 or more studies each were analyzed. The number of observed studies (O) with statistically significant results (P = 0.05 threshold) was compared with the expected number (E) under different assumptions for the magnitude of the effect size. In the main analysis, the plausible effect size of each association was the summary effect presented in the respective meta-analysis. Overall, 19 meta-analyses (all with eventually nonsignificant summary effects) had a documented excess of O over E: Typically single studies had significant effects pointing in opposite directions and early summary effects were dissipated over time. Across the whole domain, O was 235 (17.4%), while E was 164.8 (12.2%) (P < 10(-6)). The excess showed a predilection for meta-analyses with nonsignificant summary effects and between-study heterogeneity. The excess was seen for all levels of statistical significance and also for studies with borderline P values (P = 0.05-0.10). The excess of significant findings may represent significance-chasing biases in a setting of massive testing.
authors.affiliation: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
authors.forename: Fotini K
authors.initials: FK
authors.lastname: Kavvoura
authors.forename: Matthew B
authors.initials: MB
authors.lastname: McQueen
authors.forename: Muin J
authors.initials: MJ
authors.lastname: Khoury
authors.forename: Rudolph E
authors.initials: RE
authors.lastname: Tanzi
authors.forename: Lars
authors.initials: L
authors.lastname: Bertram
authors.forename: John P A
authors.initials: JP
authors.lastname: Ioannidis
mesh_terms: D000544:Alzheimer Disease / Q000453:epidemiology* / Q000235:genetics*; D015982:Bias; D020022:Genetic Predisposition to Disease / Q000453:epidemiology*; D006801:Humans
references.citation: Stat Med. 2002 Jun 15;21(11):1539-58
references.pmid: 12111919
references.citation: Nat Genet. 2001 Nov;29(3):306-9
references.pmid: 11600885
references.citation: JAMA. 1992 Jan 15;267(3):374-8
references.pmid: 1727960
references.citation: Nature. 2007 Jun 28;447(7148):1087-93
references.pmid: 17529967
references.citation: Am J Prev Med. 2007 Oct;33(4):310-7
references.pmid: 17888858
references.citation: PLoS One. 2007 Sep 05;2(9):e841
references.pmid: 17786212
references.citation: Lancet. 1991 Apr 13;337(8746):867-72
references.pmid: 1672966
references.citation: JAMA. 2004 May 26;291(20):2457-65
references.pmid: 15161896
references.citation: CMAJ. 2004 Sep 28;171(7):735-40
references.pmid: 15451835
references.citation: Am J Epidemiol. 2007 May 1;165(9):973-84
references.pmid: 17293603
references.citation: J Natl Cancer Inst. 1959 Apr;22(4):719-48
references.pmid: 13655060
references.citation: Control Clin Trials. 1986 Sep;7(3):177-88
references.pmid: 3802833
references.citation: PLoS Biol. 2005 Sep;3(9):e327
references.pmid: 16149852
references.citation: Cancer Epidemiol Biomarkers Prev. 2006 Apr;15(4):595-8
references.pmid: 16614096
references.citation: PLoS Med. 2005 Aug;2(8):e124
references.pmid: 16060722
references.citation: J Clin Epidemiol. 2005 Jun;58(6):543-9
references.pmid: 15878467
references.citation: Hum Hered. 2007;64(4):203-13
references.pmid: 17551261
references.citation: Am J Epidemiol. 2005 Aug 15;162(4):302-4
references.pmid: 16014777
references.citation: Biometrics. 1994 Dec;50(4):1088-101
references.pmid: 7786990
references.citation: Genet Med. 2002 Mar-Apr;4(2):45-61
references.pmid: 11882781
references.citation: Trends Genet. 2006 Jul;22(7):350-4
references.pmid: 16713652
references.citation: Clin Trials. 2007;4(3):245-53
references.pmid: 17715249
references.citation: Stat Med. 2001 Feb 28;20(4):641-54
references.pmid: 11223905
references.citation: PLoS Med. 2006 Dec;3(12):e520
references.pmid: 17194198
references.citation: Control Clin Trials. 2000 Oct;21(5):415-27
references.pmid: 11018560
references.citation: Mol Psychiatry. 2007 May;12(5):454-61
references.pmid: 17453061
references.citation: BMJ. 2006 Sep 16;333(7568):597-600
references.pmid: 16974018
references.citation: JAMA. 1990 Mar 9;263(10):1385-9
references.pmid: 2406472
references.citation: J Clin Epidemiol. 2006 Dec;59(12):1312-8
references.pmid: 17098574
references.citation: BMJ. 1997 Sep 13;315(7109):629-34
references.pmid: 9310563
references.citation: Nat Genet. 2005 Apr;37(4):413-7
references.pmid: 15793588
references.citation: Psychol Methods. 2006 Jun;11(2):193-206
references.pmid: 16784338
references.citation: Am J Hum Genet. 2007 Mar;80(3):531-8
references.pmid: 17273975
references.citation: Int J Epidemiol. 2008 Feb;37(1):120-32
references.pmid: 17898028
references.citation: BMJ. 2005 Jul 30;331(7511):267-70
references.pmid: 16052019
references.citation: JAMA. 1998 Jan 28;279(4):281-6
references.pmid: 9450711
references.citation: PLoS Med. 2007 Mar;4(3):e79
references.pmid: 17341129
references.citation: PLoS Med. 2005 Dec;2(12):e334
references.pmid: 16285839
references.citation: Nat Genet. 2006 Jan;38(1):3-5
references.pmid: 16468121
references.citation: Am J Hum Genet. 2007 Apr;80(4):605-15
references.pmid: 17357068
references.citation: Nat Genet. 2007 Jan;39(1):17-23
references.pmid: 17192785
references.citation: Contemp Clin Trials. 2005 Apr;26(2):244-51
references.pmid: 15837444
references.citation: Ann N Y Acad Sci. 1993 Dec 31;703:135-46; discussion 146-8
references.pmid: 8192291
references.citation: Nature. 2007 Jun 7;447(7145):655-60
references.pmid: 17554299
references.citation: J Clin Epidemiol. 2000 Nov;53(11):1119-29
references.pmid: 11106885
references.citation: Nat Genet. 2007 Sep;39(9):1045-51
references.pmid: 17728769
references.citation: Science. 2007 Jun 1;316(5829):1336-41
references.pmid: 17463249
references.citation: BMJ. 2003 Sep 6;327(7414):557-60
references.pmid: 12958120