Identifying treatment effect heterogeneity in clinical trials using subpopulations of events: STEPP.
BACKGROUND
Investigators conducting randomized clinical trials often explore treatment effect heterogeneity to assess whether treatment efficacy varies according to patient characteristics. Identifying heterogeneity is central to making informed personalized healthcare decisions. Treatment effect heterogeneity can be investigated using subpopulation treatment effect pattern plot (STEPP), a non-parametric graphical approach that constructs overlapping patient subpopulations with varying values of a characteristic. Procedures for statistical testing using subpopulation treatment effect pattern plot when the endpoint of interest is survival remain an area of active investigation.


METHODS
A STEPP analysis was used to explore patterns of absolute and relative treatment effects for varying levels of a breast cancer biomarker, Ki-67, in the phase III Breast International Group 1-98 randomized clinical trial, comparing letrozole to tamoxifen as adjuvant therapy for postmenopausal women with hormone receptor-positive breast cancer. Absolute treatment effects were measured by differences in 4-year cumulative incidence of breast cancer recurrence, while relative effects were measured by the subdistribution hazard ratio in the presence of competing risks using O-E (observed-minus-expected) methodology, an intuitive non-parametric method. While estimation of hazard ratio values based on O-E methodology has been shown, a similar development for the subdistribution hazard ratio has not. Furthermore, we observed that the subpopulation treatment effect pattern plot analysis may not produce results, even with 100 patients within each subpopulation. After further investigation through simulation studies, we observed inflation of the type I error rate of the traditional test statistic and sometimes singular variance-covariance matrix estimates that may lead to results not being produced. This is due to the lack of sufficient number of events within the subpopulations, which we refer to as instability of the subpopulation treatment effect pattern plot analysis. We introduce methodology designed to improve stability of the subpopulation treatment effect pattern plot analysis and generalize O-E methodology to the competing risks setting. Simulation studies were designed to assess the type I error rate of the tests for a variety of treatment effect measures, including subdistribution hazard ratio based on O-E estimation. This subpopulation treatment effect pattern plot methodology and standard regression modeling were used to evaluate heterogeneity of Ki-67 in the Breast International Group 1-98 randomized clinical trial.


RESULTS
We introduce methodology that generalizes O-E methodology to the competing risks setting and that improves stability of the STEPP analysis by pre-specifying the number of events across subpopulations while controlling the type I error rate. The subpopulation treatment effect pattern plot analysis of the Breast International Group 1-98 randomized clinical trial showed that patients with high Ki-67 percentages may benefit most from letrozole, while heterogeneity was not detected using standard regression modeling.


CONCLUSIONS
The STEPP methodology can be used to study complex patterns of treatment effect heterogeneity, as illustrated in the Breast International Group 1-98 randomized clinical trial. For the subpopulation treatment effect pattern plot analysis, we recommend a minimum of 20 events within each subpopulation.
Division of Oral Epidemiology, Department of Preventive and Restorative Dental Sciences, University of California, San Francisco, San Francisco, CA, USA Division of Biostatistics, Department of Epidemiology & Biostatistics, University of California, San Francisco, San Francisco, CA, USA ann.lazar@ucsf.edu.
Ann A
AA
Lazar
Carlo F. Dondena Centre for Research on Social Dynamics and Public Policies, Bocconi University, Milan, Italy.
Marco
M
Bonetti
Department of Mathematics and Statistics, University of Vermont, Burlington, VT, USA.
Bernard F
BF
Cole
Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA Department of Biostatistics and Computational Biology, Dana-Farber Cancer Institute, Boston, MA, USA.
Wai-Ki
WK
Yip
Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA Department of Biostatistics and Computational Biology, Dana-Farber Cancer Institute, Boston, MA, USA.
Richard D
RD
Gelber
D000970:Antineoplastic Agents / Q000627:therapeutic use*; D018931:Antineoplastic Agents, Hormonal / Q000627:therapeutic use*; D014408:Biomarkers, Tumor / Q000097:blood*; D001943:Breast Neoplasms / Q000188:drug therapy*; D003627:Data Interpretation, Statistical; D004311:Double-Blind Method; D005260:Female; D006801:Humans; D000077289:Letrozole; D009570:Nitriles / Q000627:therapeutic use*; D017063:Outcome Assessment, Health Care / Q000379:methods*; D012107:Research Design; D013629:Tamoxifen / Q000627:therapeutic use*; D016896:Treatment Outcome; D014230:Triazoles / Q000627:therapeutic use*
D000970:Antineoplastic Agents; D018931:Antineoplastic Agents, Hormonal; D014408:Biomarkers, Tumor; D009570:Nitriles; D014230:Triazoles; D013629:Tamoxifen; D000077289:Letrozole
Biomarker; breast cancer; competing risk; interaction; overview; permutation-based inference; personalized medicine; precision medicine; survival analysis