title: The potential for increased power from combining P-values testing the same hypothesis.
abstract: The conventional approach to hypothesis testing for formal inference is to prespecify a single test statistic thought to be optimal. However, we usually have more than one test statistic in mind for testing the null hypothesis of no treatment effect but we do not know which one is the most powerful. Rather than relying on a single p-value, combining p-values from prespecified multiple test statistics can be used for inference. Combining functions include Fisher's combination test and the minimum p-value. Using randomization-based tests, the increase in power can be remarkable when compared with a single test and Simes's method. The versatility of the method is that it also applies when the number of covariates exceeds the number of observations. The increase in power is large enough to prefer combined p-values over a single p-value. The limitation is that the method does not provide an unbiased estimator of the treatment effect and does not apply to situations when the model includes treatment by covariate interaction.
authors.affiliation: Gilead Sciences, Foster City, USA.
authors.forename: Jitendra
authors.initials: J
authors.lastname: Ganju
authors.affiliation: Gilead Sciences, Foster City, USA.
authors.forename: Guoguang
authors.initials: G
authors.lastname: Julie Ma
mesh_terms: D002986:Clinical Trials as Topic / Q000379:methods*; D003627:Data Interpretation, Statistical; D006801:Humans; D012107:Research Design; D016896:Treatment Outcome
keywords: clinical trials; increase power; multiple testing; permutation tests; randomization