The influence of display and statistical factors on the interpretation of metaanalysis results by physicians.
OBJECTIVE
The objective of this study was to determine the extent to which various factors affect the interpretation of metaanalytic results by physicians.


METHODS
A sample of 120 physicians, selected from The Royal College of Physicians and Surgeons of Canada (RCPSC), was randomly assigned to 1 of 6 groups (n = 20) created from a combination of 3 summary measures and 2 levels of disease severity. The intervention consisted of a written scenario and 4 individual displays of metaanalyses (M-A), each followed by questions related to the interpretation of results of M-A. Two final questions examined statistical familiarity/proficiency with the summary measures used.


METHODS
Analyses of variance examined main effects and interactions among 4 factors: summary measure, disease severity, effect size, and statistical consistency of the studies comprising the metaanalysis. Two outcomes were examined: interpretation of the treatment effect and confidence in the interpretation of the treatment effect.


RESULTS
Physicians were more likely to favor treatment when the results of the primary randomized, controlled trials (RCTs) were statistically homogeneous (P = 0.001) and when the overall effect size was large (P = 0.001). Also, physicians were more likely to be confident when the results were homogeneous (P = 0.001) and when effect size was large (P = 0.000). Interactions also revealed that the effect of statistical consistency of contributing to RCTs was greatest when data were presented as risk difference for treatment outcome (P = 0.026) and when effect size was small (P = 0.000).


CONCLUSIONS
The interpretation of metaanalytic displays is influenced by the overall effect size of M-A, the statistical consistency of the contributing RCTs, and interactions of these factors with display factors.
Raina
Parminder S
PS
Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada. praina@mcmaster.ca
Brehaut
Jamie C
JC
Platt
Robert W
RW
Klassen
Terry P
TP
Moher
David
D
St John
Phil
P
Bryant
Dianne
D
Viola
Ray
R
Pham
Ba
B
D000328:Adult; D000368:Aged; D000704:Analysis of Variance; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008297:Male; D015201:Meta-Analysis as Topic; D008875:Middle Aged; D010820:Physicians / Q000706:statistics & numerical data*; D016032:Randomized Controlled Trials as Topic; D012107:Research Design / Q000706:statistics & numerical data*; D012720:Severity of Illness Index; D012959:Socioeconomic Factors; D016896:Treatment Outcome