title: Comprehensive evaluations of the adverse effects of drugs: importance of appropriate study selection and data sources.
abstract: While systematic reviews and meta-analyses are at the top of the evidence hierarchy, most of the methodology has focused on assessing treatment benefit. Hence, we propose a structured framework for the initial steps of searching and identifying relevant data sources so that adverse effects can be evaluated in a comprehensive, unbiased manner. The unique methodological challenges stem from the difficulties of addressing diverse outcomes encompassing common, mild symptoms to rare, fatal events. Retrieval of the most appropriate studies should be specifically tailored to fit the nature of the adverse effects, according to the primary objective and study question. In our framework, the structure of the review takes different forms depending on whether the main aim is on scoping/hypothesis generation, or evaluating statistically the magnitude of risk (hypothesis testing), or clarifying characteristics and risk factors of the adverse effect. The wide range of data sources covering adverse effects all have distinct strengths and limitations, and selection of appropriate sources depends on characteristics of the adverse effect (e.g. background incidence and effect size of the drug, clinical presentation, time of onset after drug exposure). Reviewers need to retrieve particular study designs that are most likely to yield robust data on the adverse effects of interest, rather than rely on studies that cannot reliably detect adverse effects, and may yield 'false negatives'. Type II errors (a particular problem when evaluating rare adverse effects) can lull us into a false sense of security (e.g. wrongly concluding that there was no significant difference in harm between drug and control, with the drug erroneously judged as safe). Given the rapid rate at which methodological improvements occur, this proposed framework is by no means definitive, but aims to stimulate further debate and discussion amongst the pharmacoepidemiological and systematic review communities to reach a common consensus on the best methods.
authors.affiliation: School of Medicine, University of East Anglia, Norwich, UK.
authors.forename: Yoon K
authors.initials: YK
authors.lastname: Loke
authors.affiliation: Centre for Reviews and Dissemination, University of York, York, UK.
authors.forename: Su P
authors.initials: SP
authors.lastname: Golder
authors.affiliation: Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, The Netherlands.
authors.forename: Jan P
authors.initials: JP
authors.lastname: Vandenbroucke
keywords: adverse effects; evidence based medicine; pharmacovigilance; systematic reviews
references.citation: CMAJ. 2009 Jan 6;180(1):32-9
references.pmid: 19073651
references.citation: Arch Dis Child. 2010 Dec;95(12):1023-6
references.pmid: 20551194
references.citation: Drug Saf. 2009;32(5):441-7
references.pmid: 19419238
references.citation: Health Info Libr J. 2010 Sep;27(3):176-90
references.pmid: 20712712
references.citation: Eur Heart J. 2010 Apr;31(7):824-31
references.pmid: 20118174
references.citation: Med Care. 2007 Oct;45(10 Supl 2):S131-42
references.pmid: 17909372
references.citation: BMC Med Res Methodol. 2006 Aug 18;6:41
references.pmid: 16919159
references.citation: BMJ. 2010 Mar 18;340:c1344
references.pmid: 20299696
references.citation: Lancet. 2009 Jun 20;373(9681):2125-35
references.pmid: 19501900
references.citation: Drug Saf. 2010 Oct 1;33(10):865-78
references.pmid: 20812771
references.citation: Diabetes Care. 2007 Aug;30(8):2148-53
references.pmid: 17536074
references.citation: BMJ. 2003 Nov 22;327(7425):1222-5
references.pmid: 14630763
references.citation: Fundam Clin Pharmacol. 2002 Feb;16(1):49-56
references.pmid: 11903512
references.citation: BMJ. 2005 May 21;330(7501):1179
references.pmid: 15894554
references.citation: J Autoimmun. 2008 Feb-Mar;30(1-2):90-8
references.pmid: 18191544
references.citation: Am J Med. 2004 Oct 15;117(8):582-9
references.pmid: 15465507
references.citation: Arch Intern Med. 2001 Jan 8;161(1):125-6
references.pmid: 11146711
references.citation: Br J Clin Pharmacol. 2004 May;57(5):616-21
references.pmid: 15089815
references.citation: BMC Med Res Methodol. 2007 Jul 05;7:32
references.pmid: 17615054
references.citation: JAMA. 2007 Sep 12;298(10):1189-95
references.pmid: 17848653
references.citation: Drug Saf. 2010 Nov 1;33(11):995-1002
references.pmid: 20925437
references.citation: Drug Saf. 2006;29(12):1123-32
references.pmid: 17147459
references.citation: BMJ. 2001 Sep 8;323(7312):546
references.pmid: 11546700
references.citation: Arch Intern Med. 2009 Oct 26;169(19):1756-61
references.pmid: 19858432
references.citation: Clin Drug Investig. 2010;30(2):101-8
references.pmid: 20067328
references.citation: BMC Med Res Methodol. 2001;1:7
references.pmid: 11591220
references.citation: BMC Clin Pharmacol. 2001;1:3
references.pmid: 11591227
references.citation: J Urol. 2010 May;183(5):1693-7
references.pmid: 20299044