title: The hazards of scoring the quality of clinical trials for meta-analysis.
abstract: BACKGROUND
  Although it is widely recommended that clinical trials undergo some type of quality review, the number and variety of quality assessment scales that exist make it unclear how to achieve the best assessment.
  
  
  OBJECTIVE
  To determine whether the type of quality assessment scale used affects the conclusions of meta-analytic studies.
  
  
  METHODS
  Meta-analysis of 17 trials comparing low-molecular-weight heparin (LMWH) with standard heparin for prevention of postoperative thrombosis using 25 different scales to identify high-quality trials. The association between treatment effect and summary scores and the association with 3 key domains (concealment of treatment allocation, blinding of outcome assessment, and handling of withdrawals) were examined in regression models.
  
  
  METHODS
  Pooled relative risks of deep vein thrombosis with LMWH vs standard heparin in high-quality vs low-quality trials as determined by 25 quality scales.
  
  
  RESULTS
  Pooled relative risks from high-quality trials ranged from 0.63 (95% confidence interval [CI], 0.44-0.90) to 0.90 (95% CI, 0.67-1.21) vs 0.52 (95% CI, 0.24-1.09) to 1.13 (95% CI, 0.70-1.82) for low-quality trials. For 6 scales, relative risks of high-quality trials were close to unity, indicating that LMWH was not significantly superior to standard heparin, whereas low-quality trials showed better protection with LMWH (P<.05). Seven scales showed the opposite: high quality trials showed an effect whereas low quality trials did not. For the remaining 12 scales, effect estimates were similar in the 2 quality strata. In regression analysis, summary quality scores were not significantly associated with treatment effects. There was no significant association of treatment effects with allocation concealment and handling of withdrawals. Open outcome assessment, however, influenced effect size with the effect of LMWH, on average, being exaggerated by 35% (95% CI, 1%-57%; P= .046).
  
  
  CONCLUSIONS
  Our data indicate that the use of summary scores to identify trials of high quality is problematic. Relevant methodological aspects should be assessed individually and their influence on effect sizes explored.
authors.affiliation: Clinical Epidemiology Study Group, Institute for Medical Education, University of Berne, Switzerland.
authors.forename: P
authors.initials: P
authors.lastname: Jüni
authors.forename: A
authors.initials: A
authors.lastname: Witschi
authors.forename: R
authors.initials: R
authors.lastname: Bloch
authors.forename: M
authors.initials: M
authors.lastname: Egger
mesh_terms: D000925:Anticoagulants / Q000627:therapeutic use; D002986:Clinical Trials as Topic / Q000592:standards*; D006493:Heparin / Q000627:therapeutic use; D006495:Heparin, Low-Molecular-Weight / Q000627:therapeutic use; D006801:Humans; D015201:Meta-Analysis as Topic; D017063:Outcome Assessment, Health Care; D011183:Postoperative Complications / Q000517:prevention & control; D011786:Quality Control; D012044:Regression Analysis; D013927:Thrombosis / Q000517:prevention & control
chemical_list: D000925:Anticoagulants; D006495:Heparin, Low-Molecular-Weight; D006493:Heparin