title: The influence of display and statistical factors on the interpretation of metaanalysis results by physicians.
abstract: OBJECTIVE
  The objective of this study was to determine the extent to which various factors affect the interpretation of metaanalytic results by physicians.
  
  
  METHODS
  A sample of 120 physicians, selected from The Royal College of Physicians and Surgeons of Canada (RCPSC), was randomly assigned to 1 of 6 groups (n = 20) created from a combination of 3 summary measures and 2 levels of disease severity. The intervention consisted of a written scenario and 4 individual displays of metaanalyses (M-A), each followed by questions related to the interpretation of results of M-A. Two final questions examined statistical familiarity/proficiency with the summary measures used.
  
  
  METHODS
  Analyses of variance examined main effects and interactions among 4 factors: summary measure, disease severity, effect size, and statistical consistency of the studies comprising the metaanalysis. Two outcomes were examined: interpretation of the treatment effect and confidence in the interpretation of the treatment effect.
  
  
  RESULTS
  Physicians were more likely to favor treatment when the results of the primary randomized, controlled trials (RCTs) were statistically homogeneous (P = 0.001) and when the overall effect size was large (P = 0.001). Also, physicians were more likely to be confident when the results were homogeneous (P = 0.001) and when effect size was large (P = 0.000). Interactions also revealed that the effect of statistical consistency of contributing to RCTs was greatest when data were presented as risk difference for treatment outcome (P = 0.026) and when effect size was small (P = 0.000).
  
  
  CONCLUSIONS
  The interpretation of metaanalytic displays is influenced by the overall effect size of M-A, the statistical consistency of the contributing RCTs, and interactions of these factors with display factors.
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada. praina@mcmaster.ca
authors.forename: Parminder S
authors.initials: PS
authors.lastname: Raina
authors.forename: Jamie C
authors.initials: JC
authors.lastname: Brehaut
authors.forename: Robert W
authors.initials: RW
authors.lastname: Platt
authors.forename: Terry P
authors.initials: TP
authors.lastname: Klassen
authors.forename: David
authors.initials: D
authors.lastname: Moher
authors.forename: Phil
authors.initials: P
authors.lastname: St John
authors.forename: Dianne
authors.initials: D
authors.lastname: Bryant
authors.forename: Ray
authors.initials: R
authors.lastname: Viola
authors.forename: Ba
authors.initials: B
authors.lastname: Pham
mesh_terms: D000328:Adult; D000368:Aged; D000704:Analysis of Variance; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008297:Male; D015201:Meta-Analysis as Topic; D008875:Middle Aged; D010820:Physicians / Q000706:statistics & numerical data*; D016032:Randomized Controlled Trials as Topic; D012107:Research Design / Q000706:statistics & numerical data*; D012720:Severity of Illness Index; D012959:Socioeconomic Factors; D016896:Treatment Outcome