title: Generalizability analysis for clinical trials: a simulation study.
abstract: Subjects are rarely selected on a random basis from a well-defined patient population of interest into a clinical trial, with women, children, the elderly, and those with common comorbidities who are frequently underrepresented. Decades of clinical experience have demonstrated that the application of trial findings to individual patients is permissible by using efficacy as a measure of effectiveness and assuming that the characteristics of patients are sufficiently similar. In order to investigate this issue in greater depth, we simulated a patient population with treatment effect size of 0.5 (Cohen's d) and five covariates that included gender, health insurance, comorbidity, age, and motivation. To demonstrate how selection of patients for a clinical trial can bias the results when treatment effect varies across individuals, we created 50 nonrandom clinical trials based on this patient population and showed relative bias to range from 1.68% to 99.70%. We calculated and evaluated three indexes: C-statistics, standardized mean difference (SMD), and Tipton's index (β) of generalization for the 50 nonrandom trials. Findings indicated that (i) the ranges were 0.56-0.98, 0.23-11.17, and 0.99-0.73 for C-statistics, SMD, and β, respectively, when treatment effect bias increased from 1.68% to 99.70% and (ii) C-statistics < 0.86, SMD < 1.95, and β > 0.91 when treatment effect bias <50%. Recommendations are made using existing generalization indexes on the basis of our simulation results. An example from a real clinical trial is provided for illustration. Copyright © 2017 John Wiley & Sons, Ltd.
authors.affiliation: Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Tampa, FL, U.S.A.
authors.forename: Wei
authors.initials: W
authors.lastname: Wang
authors.affiliation: Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Tampa, FL, U.S.A.
authors.forename: Ying
authors.initials: Y
authors.lastname: Ma
authors.affiliation: Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Tampa, FL, U.S.A.
authors.forename: Yangxin
authors.identifier: 0000-0002-9792-8061
authors.initials: Y
authors.lastname: Huang
authors.affiliation: Department of Epidemiology and Biostatistics, College of Public Health, University of South Florida, Tampa, FL, U.S.A.
authors.forename: Henian
authors.identifier: 0000-0003-1297-4622
authors.initials: H
authors.lastname: Chen
mesh_terms: D000367:Age Factors; D056808:Biostatistics; D002986:Clinical Trials as Topic / Q000706:statistics & numerical data*; D015897:Comorbidity; D003198:Computer Simulation; D005260:Female; D006801:Humans; D007348:Insurance, Health; D008297:Male; D009042:Motivation; D018579:Patient Selection; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data; D015983:Selection Bias; D012737:Sex Factors
keywords: bias; clinical trial; effect size; generalizability; simulation