title: Trial sequential analysis may be insufficient to draw firm conclusions regarding statistically significant treatment differences using observed intervention effects: a case study of meta-analyses of multiple myeloma trials.
abstract: Trial sequential analysis (TSA) has been proposed as a method to assess the risk of random error in cumulative meta-analysis (MA), which increases due to repeated significance testing. The aim of TSA is to assist researchers from wrongly concluding treatment differences in the absence of a benefit (i.e. true versus false positive). Similar to monitoring boundaries applied in individual randomized controlled trials, recent literature has advocated the use of TSA for assessing the conclusiveness of results from MAs to determine the requirement for future studies in case of true positive results. While this may be desirable, we present empirical evidence from a recent systematic review to demonstrate that the use of TSA may lead to a premature declaration of statistically significant treatment difference, when further accumulated evidence suggested otherwise. Using all apparently conclusive MAs in multiple-myeloma, we empirically studied under what thresholds for the risk ratio reduction and power a true positive result becomes false positive. We recommend that the conclusion of significant treatment differences in cumulative MA should be weighed against acceptable thresholds regarding the type I error, power and apriori specified clinically meaningful treatment difference.
authors.affiliation: Center for Evidence Based Medicine and Health Outcomes Research, University of South Florida, Tampa, FL, USA. bmiladin@health.usf.edu
authors.forename: Branko
authors.initials: B
authors.lastname: Miladinovic
authors.forename: Ambuj
authors.initials: A
authors.lastname: Kumar
authors.forename: Iztok
authors.initials: I
authors.lastname: Hozo
authors.forename: Helen
authors.initials: H
authors.lastname: Mahony
authors.forename: Benjamin
authors.initials: B
authors.lastname: Djulbegovic
mesh_terms: D015982:Bias; D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D009101:Multiple Myeloma / Q000628:therapy*; D016032:Randomized Controlled Trials as Topic; D012306:Risk; D018401:Sample Size