Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach.
OBJECTIVE
At least part of the failure in the transition from experimental to clinical studies in stroke has been attributed to the imprecision introduced by problems in the design of experimental stroke studies. Using a metaepidemiologic approach, we addressed the effect of randomization, blinding, and use of comorbid animals on the estimate of how effectively therapeutic interventions reduce infarct size.


METHODS
Electronic and manual searches were performed to identify meta-analyses that described interventions in experimental stroke. For each meta-analysis thus identified, a reanalysis was conducted to estimate the impact of various quality items on the estimate of efficacy, and these estimates were combined in a meta-meta-analysis to obtain a summary measure of the impact of the various design characteristics.


RESULTS
Thirteen meta-analyses that described outcomes in 15,635 animals were included. Studies that included unblinded induction of ischemia reported effect sizes 13.1% (95% CI, 26.4% to 0.2%) greater than studies that included blinding, and studies that included healthy animals instead of animals with comorbidities overstated the effect size by 11.5% (95% CI, 21.2% to 1.8%). No significant effect was found for randomization, blinded outcome assessment, or high aggregate CAMARADES quality score.


CONCLUSIONS
We provide empirical evidence of bias in the design of studies, with studies that included unblinded induction of ischemia or healthy animals overestimating the effectiveness of the intervention. This bias could account for the failure in the transition from bench to bedside of stroke therapies.
Crossley
Nicolas A
NA
Center for Stroke Research, Department of Experimental Neurology, Charité Universitätsmedizin Berlin, 10098 Berlin, Germany.
Sena
Emily
E
Goehler
Jos
J
Horn
Jannekke
J
van der Worp
Bart
B
Bath
Philip M W
PM
Macleod
Malcolm
M
Dirnagl
Ulrich
U
D000818:Animals; D015982:Bias; D035843:Biomedical Research / Q000592:standards; D002545:Brain Ischemia / Q000150:complications* / Q000188:drug therapy*; D002544:Cerebral Infarction / Q000209:etiology* / Q000473:pathology*; D015897:Comorbidity; D004812:Epidemiologic Methods; D015201:Meta-Analysis as Topic; D018696:Neuroprotective Agents / Q000627:therapeutic use*; D011897:Random Allocation; D015203:Reproducibility of Results; D012106:Research / Q000592:standards*; D016037:Single-Blind Method; D016896:Treatment Outcome
D018696:Neuroprotective Agents