Randomized trials, generalizability, and meta-analysis: graphical insights for binary outcomes.
BACKGROUND
Randomized trials stochastically answer the question. "What would be the effect of treatment on outcome if one turned back the clock and switched treatments in the given population?" Generalizations to other subjects are reliable only if the particular trial is performed on a random sample of the target population. By considering an unobserved binary variable, we graphically investigate how randomized trials can also stochastically answer the question, "What would be the effect of treatment on outcome in a population with a possibly different distribution of an unobserved binary baseline variable that does not interact with treatment in its effect on outcome?"


METHODS
For three different outcome measures, absolute difference (DIF), relative risk (RR), and odds ratio (OR), we constructed a modified BK-Plot under the assumption that treatment has the same effect on outcome if either all or no subjects had a given level of the unobserved binary variable. (A BK-Plot shows the effect of an unobserved binary covariate on a binary outcome in two treatment groups; it was originally developed to explain Simpsons's paradox.)


RESULTS
For DIF and RR, but not OR, the BK-Plot shows that the estimated treatment effect is invariant to the fraction of subjects with an unobserved binary variable at a given level.


CONCLUSIONS
The BK-Plot provides a simple method to understand generalizability in randomized trials. Meta-analyses of randomized trials with a binary outcome that are based on DIF or RR, but not OR, will avoid bias from an unobserved covariate that does not interact with treatment in its effect on outcome.
Baker
Stuart G
SG
Biometry Research Group, Division of Cancer Prevention, National Cancer Institute, USA. sb16i@nih.gov
Kramer
Barnett S
BS
D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D016017:Odds Ratio; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D012306:Risk; D016896:Treatment Outcome