title: Proper interpretation of chronic toxicity studies and their statistics: A critique of "Which level of evidence does the US National Toxicology Program provide? Statistical considerations using the Technical Report 578 on Ginkgo biloba as an example".
abstract: A recent article by Gaus (2014) demonstrates a serious misunderstanding of the NTP's statistical analysis and interpretation of rodent carcinogenicity data as reported in Technical Report 578 (Ginkgo biloba) (NTP, 2013), as well as a failure to acknowledge the abundant literature on false positive rates in rodent carcinogenicity studies. The NTP reported Ginkgo biloba extract to be carcinogenic in mice and rats. Gaus claims that, in this study, 4800 statistical comparisons were possible, and that 209 of them were statistically significant (p<0.05) compared with 240 (4800×0.05) expected by chance alone; thus, the carcinogenicity of Ginkgo biloba extract cannot be definitively established. However, his assumptions and calculations are flawed since he incorrectly assumes that the NTP uses no correction for multiple comparisons, and that significance tests for discrete data operate at exactly the nominal level. He also misrepresents the NTP's decision making process, overstates the number of statistical comparisons made, and ignores the fact that the mouse liver tumor effects were so striking (e.g., p<0.0000000000001) that it is virtually impossible that they could be false positive outcomes. Gaus' conclusion that such obvious responses merely "generate a hypothesis" rather than demonstrate a real carcinogenic effect has no scientific credibility. Moreover, his claims regarding the high frequency of false positive outcomes in carcinogenicity studies are misleading because of his methodological misconceptions and errors.
authors.affiliation: National Institute of Environmental Health Science, P.O. Box 12233, Research Triangle Park, NC 27709, United States; J.K. Haseman Consulting, 1054 Tacketts Pond Drive, Raleigh, NC 27614-7886, United States; Errol Zeiger Consulting, 800 Indian Springs Road, Chapel Hill, NC 27514, United States. Electronic address: kissling@niehs.nih.gov.
authors.forename: Grace E
authors.initials: GE
authors.lastname: Kissling
authors.affiliation: National Institute of Environmental Health Science, P.O. Box 12233, Research Triangle Park, NC 27709, United States; J.K. Haseman Consulting, 1054 Tacketts Pond Drive, Raleigh, NC 27614-7886, United States; Errol Zeiger Consulting, 800 Indian Springs Road, Chapel Hill, NC 27514, United States.
authors.forename: Joseph K
authors.initials: JK
authors.lastname: Haseman
authors.affiliation: National Institute of Environmental Health Science, P.O. Box 12233, Research Triangle Park, NC 27709, United States; J.K. Haseman Consulting, 1054 Tacketts Pond Drive, Raleigh, NC 27614-7886, United States; Errol Zeiger Consulting, 800 Indian Springs Road, Chapel Hill, NC 27514, United States.
authors.forename: Errol
authors.initials: E
authors.lastname: Zeiger
mesh_terms: D000818:Animals; D015197:Carcinogenicity Tests; D003627:Data Interpretation, Statistical; D005260:Female; D020441:Ginkgo biloba / Q000633:toxicity*; D008297:Male; D051379:Mice; D051381:Rats; D036401:Toxicity Tests, Chronic
keywords: False positive; Ginkgo biloba; NTP carcinogenicity; Statistical analysis