Post hoc power estimation in large-scale multiple testing problems.
BACKGROUND
The statistical power or multiple Type II error rate in large-scale multiple testing problems as, for example, in gene expression microarray experiments, depends on typically unknown parameters and is therefore difficult to assess a priori. However, it has been suggested to estimate the multiple Type II error rate post hoc, based on the observed data.


METHODS
We consider a class of post hoc estimators that are functions of the estimated proportion of true null hypotheses among all hypotheses. Numerous estimators for this proportion have been proposed and we investigate the statistical properties of the derived multiple Type II error rate estimators in an extensive simulation study.


RESULTS
The performance of the estimators in terms of the mean squared error depends sensitively on the distributional scenario. Estimators based on empirical distributions of the null hypotheses are superior in the presence of strongly correlated test statistics.


BACKGROUND
R-code to compute all considered estimators based on P-values and supplementary material is available on the authors web page http://statistics.msi.meduniwien.ac.at/index.php?page=pageszfnr.
Zehetmayer
Sonja
S
Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, A-1090 Vienna, Austria.
Posch
Martin
M
D000465:Algorithms; D014408:Biomarkers, Tumor / Q000737:chemistry; D020869:Gene Expression Profiling / Q000379:methods; D023281:Genomics / Q000379:methods*; D006801:Humans; D015233:Models, Statistical; D020411:Oligonucleotide Array Sequence Analysis / Q000379:methods*
D014408:Biomarkers, Tumor