title: When is a meta-analysis conclusive? A guide to Trial Sequential Analysis with an example of remote ischemic preconditioning for renoprotection in patients undergoing cardiac surgery.
abstract: Regardless of whether a randomized trial finds a statistically significant effect for an intervention or not, readers often wonder if the trial was large enough to be conclusive. To answer this question, we can estimate the required sample size for a trial by considering how commonly the outcome occurs, the smallest effect of clinical importance and the acceptable risk of falsely detecting or rejecting that effect. But when is a meta-analysis conclusive? We explain and illustrate the interpretation of Trial Sequential Analysis (TSA), a method increasingly used to answer this question. We conducted a conventional meta-analysis which suggested that, in adults undergoing cardiac surgery, remote ischemic preconditioning does not provide a statistically significant reduction in acute kidney injury (AKI) [12 trials, 4230 patients; relative risk 0.87 (95% confidence interval 0.74-1.02); P = 0.08; I2= 35%] or the risk of receiving acute dialysis [5 trials, 2111 patients; relative risk 1.15 (95% confidence interval 0.42-3.19); P = 0.78; I2 = 59%]. TSA demonstrates that as little as a 20% relative risk reduction in AKI is unlikely. Reliably finding effects on acute dialysis and smaller effects on AKI would require much more evidence. Notably, conventional meta-analyses conducted at one of the two earlier time points may have prematurely declared a statistically significant reduction in AKI, even though at no point in the TSA was there sufficient evidence to support such an effect. With this and other examples, we demonstrate that the TSA can prevent premature conclusions from meta-analyses.
authors.affiliation: London Kidney Clinical Research Unit, London Health Sciences Centre, Room ELL-101, Westminster, 800 Commissioners Road East, London, ON, Canada.|Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada.|Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
authors.forename: Pavel S
authors.initials: PS
authors.lastname: Roshanov
authors.affiliation: St. George's University of London, London, UK.
authors.forename: Brittany B
authors.initials: BB
authors.lastname: Dennis
authors.affiliation: Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
authors.forename: Nicholas
authors.initials: N
authors.lastname: Pasic
authors.affiliation: London Kidney Clinical Research Unit, London Health Sciences Centre, Room ELL-101, Westminster, 800 Commissioners Road East, London, ON, Canada.|Institute for Clinical Evaluative Sciences, London, ON, Canada.|Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
authors.forename: Amit X
authors.initials: AX
authors.lastname: Garg
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada.|Department of Medicine, McMaster University, Hamilton, ON, Canada.|Population Health Research Institute, Hamilton, ON, Canada.
authors.forename: Michael
authors.initials: M
authors.lastname: Walsh
mesh_terms: D058186:Acute Kidney Injury / Q000517:prevention & control*; D000328:Adult; D006348:Cardiac Surgical Procedures / Q000009:adverse effects*; D002986:Clinical Trials as Topic / Q000592:standards*; D003627:Data Interpretation, Statistical; D006801:Humans; D019194:Ischemic Preconditioning / Q000379:methods*; D015201:Meta-Analysis as Topic; D012307:Risk Factors
keywords: information size; meta-analysis; monitoring boundaries; remote ischemic preconditioning; sample size; trial sequential analysis