title: Valid randomization-based p-values for partially post hoc subgroup analyses.
abstract: By 'partially post-hoc' subgroup analyses, we mean analyses that compare existing data from a randomized experiment-from which a subgroup specification is derived-to new, subgroup-only experimental data. We describe a motivating example in which partially post hoc subgroup analyses instigated statistical debate about a medical device's efficacy. We clarify the source of such analyses' invalidity and then propose a randomization-based approach for generating valid posterior predictive p-values for such partially post hoc subgroups. Lastly, we investigate the approach's operating characteristics in a simple illustrative setting through a series of simulations, showing that it can have desirable properties under both null and alternative hypotheses.
authors.affiliation: Department of Statistics, Harvard University, 1 Oxford St., Cambridge, MA, 02138, U.S.A.
authors.forename: Joseph J
authors.identifier: http://orcid.org/0000-0001-8016-2592
authors.initials: JJ
authors.lastname: Lee
authors.affiliation: Department of Statistics, Harvard University, 1 Oxford St., Cambridge, MA, 02138, U.S.A.
authors.forename: Donald B
authors.initials: DB
authors.lastname: Rubin
mesh_terms: D001699:Biometry; D003198:Computer Simulation; D004864:Equipment and Supplies; D005782:Gels / Q000627:therapeutic use; D006801:Humans; D020370:Osteoarthritis, Knee / Q000188:drug therapy; D018579:Patient Selection; D016032:Randomized Controlled Trials as Topic / Q000379:methods*; D012107:Research Design; D014481:United States; D014486:United States Food and Drug Administration
chemical_list: D005782:Gels
keywords: Fisher randomization test; causal inference; multiple comparisons; posterior predictive p-value; statistical significance