title: Quality appraisal in systematic reviews of public health interventions: an empirical study on the impact of choice of tool on meta-analysis.
abstract: BACKGROUND
  Systematic reviews are a cornerstone of evidence-based public health. The method of appraising the quality of different intervention and observational study designs in such reviews remains an important challenge. This article examines the applicability of selected quality appraisal tools (QATs) and the impact of choice of tool on the meta-analysis of a published systematic review.
  
  
  METHODS
  The authors selected a systematic review on the effectiveness of hand washing with soap in preventing diarrhoea, covering a range of epidemiological study designs. 6 QATs were used to assess 13 studies meeting their inclusion criteria; component sections/questions were coded numerically to derive a summary score between -1 (low quality) and +1 (high quality) for each QAT and study. Heterogeneity in study quality was evaluated graphically using traffic light schemes and spider charts. Random effects meta-analysis was undertaken for all studies; sensitivity analyses for each QAT included only those studies with a score of 0 or above.
  
  
  RESULTS
  The authors found substantial heterogeneity in summary scores for a given study. Their main meta-analysis yielded an OR of 0.60 (95% CI 0.47 to 0.77) with most sensitivity analyses giving similar pooled effect sizes with wider CIs.
  
  
  CONCLUSIONS
  The six QATs differ greatly in applicability across study designs, approach to quality appraisal (ie, scale vs checklist, presence/absence of summary score), coverage of domains and quality of component questions and answers. Learning from advantages and disadvantages of each QAT, we recommend research into the development of a reliable QAT with a broad applicability across study designs.
authors.affiliation: Institute for Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistrasse 15, 81377 Munich, Germany.
authors.forename: Peer H
authors.initials: PH
authors.lastname: Voss
authors.forename: Eva A
authors.initials: EA
authors.lastname: Rehfuess
mesh_terms: D003967:Diarrhea / Q000517:prevention & control; D036262:Empirical Research; D019317:Evidence-Based Medicine / Q000295:instrumentation* / Q000592:standards; D006235:Hand Disinfection; D006801:Humans; D015201:Meta-Analysis as Topic; D011634:Public Health; D011786:Quality Control; D012107:Research Design / Q000592:standards*; D012196:Review Literature as Topic; D012915:Soaps
chemical_list: D012915:Soaps