title: Comparison of pooled risk estimates for adverse effects from different observational study designs: methodological overview.
abstract: BACKGROUND
  A diverse range of study designs (e.g. case-control or cohort) are used in the evaluation of adverse effects. We aimed to ascertain whether the risk estimates from meta-analyses of case-control studies differ from that of other study designs.
  
  
  METHODS
  Searches were carried out in 10 databases in addition to reference checking, contacting experts, and handsearching key journals and conference proceedings. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from case-control studies could be directly compared with the pooled estimate for the same adverse effect arising from other types of observational studies.
  
  
  RESULTS
  We included 82 meta-analyses. Pooled estimates of harm from the different study designs had 95% confidence intervals that overlapped in 78/82 instances (95%). Of the 23 cases of discrepant findings (significant harm identified in meta-analysis of one type of study design, but not with the other study design), 16 (70%) stemmed from significantly elevated pooled estimates from case-control studies. There was associated evidence of funnel plot asymmetry consistent with higher risk estimates from case-control studies. On average, cohort or cross-sectional studies yielded pooled odds ratios 0.94 (95% CI 0.88-1.00) times lower than that from case-control studies.
  
  
  CONCLUSIONS
  Empirical evidence from this overview indicates that meta-analysis of case-control studies tend to give slightly higher estimates of harm as compared to meta-analyses of other observational studies. However it is impossible to rule out potential confounding from differences in drug dose, duration and populations when comparing between study designs.
authors: Centre for Reviews and Dissemination (CRD), University of York, York, United Kingdom.
authors: Su
authors: S
authors: Golder
authors: Yoon K
authors: YK
authors: Loke
authors: Martin
authors: M
authors: Bland
mesh_terms: D000894:Anti-Inflammatory Agents, Non-Steroidal / Q000009:adverse effects / Q000627:therapeutic use; D016022:Case-Control Studies; D003278:Contraceptives, Oral, Hormonal / Q000009:adverse effects; D020249:Hormone Replacement Therapy / Q000009:adverse effects; D006801:Humans; D015201:Meta-Analysis as Topic; D064887:Observational Studies as Topic; D016017:Odds Ratio; D012107:Research Design; D018570:Risk Assessment / Q000379:methods
chemical_list: D000894:Anti-Inflammatory Agents, Non-Steroidal; D003278:Contraceptives, Oral, Hormonal
references: J Rheumatol. 2002 Apr;29(4):804-12
references: 11950025
references: Am J Hypertens. 1997 Nov;10(11):1191-200
references: 9397236
references: Pain. 2000 Mar;85(1-2):169-82
references: 10692616
references: J Clin Oncol. 2001 Aug 15;19(16):3685-91
references: 11504750
references: Ann Intern Med. 2005 Jun 21;142(12 Pt 2):1090-9
references: 15968034
references: Ann Surg. 2006 Nov;244(5):668-76
references: 17060757
references: Int J Clin Pract Suppl. 2003 Apr;(135):43-9
references: 12723747
references: Pediatrics. 2007 Jan;119(1):e275-83
references: 17178922
references: CMAJ. 2009 Jan 6;180(1):32-9
references: 19073651
references: Acta Obstet Gynecol Scand. 2009;88(3):261-6
references: 19172422
references: Diabetes Care. 2007 Aug;30(8):2148-53
references: 17536074
references: J Prosthet Dent. 2000 Jul;84(1):8-16
references: 10898835
references: J Clin Epidemiol. 2007 Jan;60(1):18-28
references: 17161750
references: Br J Anaesth. 1997 Jun;78(6):642-51
references: 9215013
references: Med J Aust. 1995 May 1;162(9):491-3
references: 7746209
references: Pharmacotherapy. 1998 Nov-Dec;18(6):1290-7
references: 9855329
references: Lancet. 2003 Apr 5;361(9364):1159-67
references: 12686037
references: Aust N Z J Public Health. 1999 Feb;23(1):34-40
references: 10083687
references: Lancet. 2005 Jan 1-7;365(9453):82-93
references: 15639683
references: Community Eye Health. 1998;11(28):57-8
references: 17492047
references: Evid Based Med. 2008 Jun;13(3):67-9
references: 18515615
references: Br J Surg. 1998 Feb;85(2):171-8
references: 9501809
references: Expert Rev Pharmacoecon Outcomes Res. 2002 Aug;2(4):367-79
references: 19807443
references: Neurology. 1998 Aug;51(2):411-8
references: 9710012
references: Arch Dis Child. 2002 Dec;87(6):493-4
references: 12456546
references: Clin Gastroenterol Hepatol. 2009 Aug;7(8):874-81
references: 19558997
references: Br J Clin Pharmacol. 2004 May;57(5):616-21
references: 15089815
references: CMAJ. 1999 Apr 6;160(7):987-8
references: 10207335
references: BMJ. 2008 May 31;336(7655):1227-31
references: 18495631
references: Congenit Anom (Kyoto). 2005 Jun;45(2):44-51
references: 15904431
references: Stat Med. 2001 Dec 15;20(23):3557-69
references: 11746337
references: J Intern Med. 1995 Jul;238(1):31-7
references: 7608644
references: CMAJ. 2006 Feb 28;174(5):635-41
references: 16505459
references: Heart. 2003 Jul;89(7):767-72
references: 12807853
references: CMAJ. 2008 Jun 17;178(13):1669-78
references: 18559804
references: JAMA. 2001 Aug 15;286(7):821-30
references: 11497536
references: Arch Intern Med. 2004 Apr 12;164(7):741-7
references: 15078643
references: J Am Geriatr Soc. 1999 Jan;47(1):30-9
references: 9920227
references: J Clin Epidemiol. 2008 Oct;61(10):991-6
references: 18538991
references: Health Technol Assess. 1998;2(13):i-iv, 1-124
references: 9793791
references: BMJ. 1998 Sep 26;317(7162):839-43
references: 9748174
references: JAMA. 1999 Mar 3;281(9):824-9
references: 10071004
references: Ultrasound Obstet Gynecol. 2009 May;33(5):599-608
references: 19291813
references: Reg Anesth Pain Med. 2002 Nov-Dec;27(6):612-7
references: 12430114
references: Drugs Aging. 1999 Jan;14(1):55-61
references: 10069408
references: Semin Hematol. 2008 Jul;45(3):181-8
references: 18582625
references: CMAJ. 2006 Feb 28;174(5):645-6
references: 16505461
references: JAMA. 2000 Jul 5;284(1):72-8
references: 10872016
references: Arch Intern Med. 1992 Jun;152(6):1289-95
references: 1534651
references: Int J Fertil Womens Med. 2005 Nov-Dec;50(6):259-66
references: 16526416
references: Obstet Gynecol. 1995 Feb;85(2):304-13
references: 7824251
references: Eur J Heart Fail. 2008 Nov;10(11):1102-7
references: 18760966
references: J Clin Epidemiol. 2010 May;63(5):502-12
references: 18823754
references: Clin Exp Allergy. 2008 Apr;38(4):634-42
references: 18266879
references: JAMA. 2006 Oct 4;296(13):1633-44
references: 16968831
references: J Am Geriatr Soc. 1999 Jan;47(1):40-50
references: 9920228
references: N Engl J Med. 2000 Jun 22;342(25):1887-92
references: 10861325
references: Drug Saf. 2007;30(2):171-84
references: 17253881
references: Drug Saf. 2008;31(9):775-88
references: 18707192
references: Arch Intern Med. 2001 Apr 23;161(8):1041-6
references: 11322836
references: JAMA. 1994 May 25;271(20):1615-9
references: 8182815
references: Health Technol Assess. 2010 Mar;14(16):1-165, iii
references: 20338119
references: J Clin Epidemiol. 2013 Mar;66(3):253-60
references: 23347849
references: PLoS Med. 2011 May;8(5):e1001026
references: 21559325
references: Obstet Gynecol. 1998 Sep;92(3):472-9
references: 9721791
references: Arch Intern Med. 1997 Jul 28;157(14):1522-30
references: 9236553
references: Breast Cancer Res. 2005;7(4):R535-40
references: 15987460
references: Aliment Pharmacol Ther. 2007 Oct 15;26(8):1089-99
references: 17894651
references: Obstet Gynecol. 2005 Jul;106(1):173-9
references: 15994634
references: Ther Adv Drug Saf. 2011 Apr;2(2):59-68
references: 25083202
references: Int J Cancer. 2007 Feb 15;120(4):833-43
references: 17131313
references: J Card Surg. 2007 May-Jun;22(3):247-56
references: 17488432
references: Health Technol Assess. 2000;4(34):1-154
references: 11134917
references: J Am Geriatr Soc. 1999 Jun;47(6):749-54
references: 10366179
references: Hosp Med. 1998 Jun;59(6):499-501
references: 9775281
references: Eur J Epidemiol. 2006;21(4):267-77
references: 16685577
references: Ann Rheum Dis. 2007 Oct;66(10):1296-304
references: 17344246
references: Ann Intern Med. 2006 Feb 21;144(4):298-300
references: 16490917
references: Arch Intern Med. 2009 Nov 23;169(21):1952-60
references: 19933955
references: J Clin Epidemiol. 2010 Oct;63(10):1071-81
references: 20457510
references: BMJ. 2003 Sep 6;327(7414):557-60
references: 12958120