Nonrandomized quality improvement intervention trials might overstate the strength of causal inference of their findings.
OBJECTIVE
To assess the strength of causal inferences reported in randomized and nonrandomized evaluations of quality improvement (QI) interventions in relation to the study design and the direction of results for the primary outcomes.


METHODS
We searched 11 journals for QI intervention studies that aimed to change clinician behavior. Statements that addressed the causal inference between intervention and outcomes were extracted and were rated by 34 researchers for the strength of causality.


RESULTS
We found 38 randomized controlled trials (RCTs) and 35 non-RCTs, and extracted 68 quotes from the abstracts and 139 from the main text. A significant interaction was found between study design and direction of results for the abstract quotes (P=0.022). The ratings for non-RCTs were higher when the results were mixed, but for RCTs, they were higher if the results were positive or no effect, although none of the differences were statistically significant at alpha=0.05 after adjusting for multiple comparisons. For the main text quotes, the causality rating was higher by 0.43 for RCTs than for non-RCTs after adjusting for the direction of results (P<0.001).


CONCLUSIONS
Authors might have overstated the strength of causal inference in the abstracts of non-RCTs, but appeared to report causality appropriately in the main text.
Li
Linda C
LC
Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada. lli@arthritisresearch.ca
Moja
Lorenzo
L
Romero
Alberto
A
Sayre
Eric C
EC
Grimshaw
Jeremy M
JM
D000328:Adult; D015984:Causality; D002983:Clinical Competence; D018849:Controlled Clinical Trials as Topic / Q000379:methods*; D005260:Female; D006282:Health Personnel / Q000592:standards; D006801:Humans; D008297:Male; D008875:Middle Aged; D011594:Psychometrics; D016032:Randomized Controlled Trials as Topic / Q000379:methods; D012107:Research Design; D018401:Sample Size