title: Randomized Trials Built on Sand: Examples from COPD, Hormone Therapy, and Cancer.
abstract: The randomized controlled trial is the fundamental study design to evaluate the effectiveness of medications and receive regulatory approval. Observational studies, on the other hand, are essential to address post-marketing drug safety issues but have also been used to uncover new indications or new benefits for already marketed drugs. Hormone replacement therapy (HRT) for instance, effective for menopausal symptoms, was reported in several observational studies during the 1980s and 1990s to also significantly reduce the incidence of coronary heart disease. This claim was refuted in 2002 by the large-scale Women's Health Initiative randomized trial. An example of a new indication for an old drug is that of metformin, an anti-diabetic medication, which is being hailed as a potential anti-cancer agent, primarily on the basis of several recent observational studies that reported impressive reductions in cancer incidence and mortality with its use. These observational studies have now sparked the conduct of large-scale randomized controlled trials currently ongoing in cancer. We show in this paper that the spectacular effects on new indications or new outcomes reported in many observational studies in chronic obstructive pulmonary disease (COPD), HRT, and cancer are the result of time-related biases, such as immortal time bias, that tend to seriously exaggerate the benefits of a drug and that eventually disappear with the proper statistical analysis. In all, while observational studies are central to assess the effects of drugs, their proper design and analysis are essential to avoid bias. The scientific evidence on the potential beneficial effects in new indications of existing drugs will need to be more carefully assessed before embarking on long and expensive unsubstantiated trials.
authors.affiliation: Center for Clinical Epidemiology, Lady Davis Institute, Jewish General Hospital, and Departments of Epidemiology and Biostatistics, and of Medicine, McGill University, Montreal, Canada.
authors.forename: Samy
authors.initials: S
authors.lastname: Suissa
keywords: Cohort studies; drug effectiveness; drug indications; observational studies; randomized controlled trials; scientific evidence
references.citation: Cancer Prev Res (Phila). 2010 Nov;3(11):1451-61
references.pmid: 20947488
references.citation: Am J Med. 2010 Nov;123(11):1001-6
references.pmid: 20870201
references.citation: N Engl J Med. 2007 Feb 22;356(8):775-89
references.pmid: 17314337
references.citation: Pharmacoepidemiol Drug Saf. 2007 Mar;16(3):241-9
references.pmid: 17252614
references.citation: Thorax. 1998 Jun;53(6):477-82
references.pmid: 9713447
references.citation: Natl Vital Stat Rep. 2010 Dec;59(2):1-52
references.pmid: 25073655
references.citation: Cancer Res. 2007 Nov 15;67(22):10804-12
references.pmid: 18006825
references.citation: BMJ. 2010 Mar 12;340:b5087
references.pmid: 20228141
references.citation: Diabetologia. 2010 Aug;53(8):1631-7
references.pmid: 20407744
references.citation: Lancet. 2007 Sep 1;370(9589):765-73
references.pmid: 17765526
references.citation: Circulation. 2006 Apr 25;113(16):1950-7
references.pmid: 16618816
references.citation: Am J Epidemiol. 2008 Feb 15;167(4):492-9
references.pmid: 18056625
references.citation: Oncogene. 2008 Jun 5;27(25):3576-86
references.pmid: 18212742
references.citation: Diabetes Care. 2009 Jan;32(1):193-203
references.pmid: 18945920
references.citation: BMC Cancer. 2011 Jan 18;11:20
references.pmid: 21241523
references.citation: Arch Intern Med. 1993 Jan 11;153(1):73-9
references.pmid: 8422201
references.citation: Arch Intern Med. 1997 Oct 27;157(19):2181-7
references.pmid: 9342994
references.citation: Am J Respir Crit Care Med. 2002 Oct 15;166(8):1084-91
references.pmid: 12379552
references.citation: Am J Cardiol. 1997 Apr 1;79(7):847-50
references.pmid: 9104892
references.citation: Lancet. 1998 Mar 14;351(9105):773-80
references.pmid: 9519948
references.citation: Am J Respir Crit Care Med. 2003 Jul 1;168(1):49-53
references.pmid: 12663327
references.citation: Eur Respir J. 2009 Jul;34(1):13-6
references.pmid: 19567599
references.citation: Obstet Gynecol. 1995 Jan;85(1):6-10
references.pmid: 7800326
references.citation: Diabetologia. 2011 Sep;54(9):2254-62
references.pmid: 21614572
references.citation: JAMA. 2002 Jul 17;288(3):321-33
references.pmid: 12117397
references.citation: N Engl J Med. 2000 Dec 28;343(26):1902-9
references.pmid: 11136260
references.citation: Cancer Epidemiol Biomarkers Prev. 2009 Mar;18(3):701-5
references.pmid: 19240238
references.citation: N Engl J Med. 2007 Jan 4;356(1):29-38
references.pmid: 17202453
references.citation: J Allergy Clin Immunol. 2005 Apr;115(4):714-9
references.pmid: 15805989
references.citation: Am J Respir Crit Care Med. 2007 Jan 15;175(2):144-9
references.pmid: 17053207
references.citation: JAMA. 2005 Dec 21;294(23):2989-95
references.pmid: 16414946
references.citation: Eur Respir J. 2008 May;31(5):927-33
references.pmid: 18216056
references.citation: Am J Respir Crit Care Med. 2001 Aug 15;164(4):580-4
references.pmid: 11520719
references.citation: Ann Oncol. 2011 Dec;22(12):2640-2645
references.pmid: 21415239
references.citation: J Am Coll Cardiol. 1997 Jan;29(1):1-5
references.pmid: 8996287
references.citation: Lancet. 1999 May 29;353(9167):1819-23
references.pmid: 10359405
references.citation: Am J Respir Crit Care Med. 2005 Aug 15;172(4):460-4
references.pmid: 15901610
references.citation: BMJ. 2005 Jun 4;330(7503):1304-5
references.pmid: 15849206
references.citation: Diabetes Care. 2010 Jun;33(6):1304-8
references.pmid: 20299480
references.citation: Lancet. 2003 Feb 8;361(9356):449-56
references.pmid: 12583942
references.citation: Diabetes Care. 2006 Feb;29(2):254-8
references.pmid: 16443869
references.citation: Eur Respir J. 2003 Dec;22(6):912-9
references.pmid: 14680078
references.citation: J Clin Oncol. 2009 Jul 10;27(20):3271-3
references.pmid: 19487373
references.citation: Eur Respir J. 2003 Jan;21(1):74-81
references.pmid: 12570112
references.citation: Cancer Res. 2006 Nov 1;66(21):10269-73
references.pmid: 17062558
references.citation: BMJ. 2010 Jun 03;340:c2519
references.pmid: 20525678
references.citation: BMJ. 2000 May 13;320(7245):1297-303
references.pmid: 10807619
references.citation: Diabetes Care. 2011 Apr;34(4):923-9
references.pmid: 21447664
references.citation: Am J Respir Crit Care Med. 2006 Feb 15;173(4):464; author reply 464-5
references.pmid: 16467177
references.citation: Eur Respir J. 2004 Mar;23(3):391-5
references.pmid: 15065827
references.citation: N Engl J Med. 1999 Jun 24;340(25):1948-53
references.pmid: 10379018
references.citation: BMJ. 2010 Feb 05;340:c249
references.pmid: 20139216
references.citation: N Engl J Med. 2000 Aug 3;343(5):332-6
references.pmid: 10922423
references.citation: Kidney Int. 2006 Mar;69(5):913-9
references.pmid: 16518351
references.citation: Chest. 1996 May;109(5):1156-62
references.pmid: 8625660
references.citation: Eur Respir J. 2006 Feb;27(2):397-412
references.pmid: 16452599
references.citation: Diabetes Care. 2012 Dec;35(12):2665-73
references.pmid: 23173135
references.citation: Annu Rev Public Health. 1998;19:55-72
references.pmid: 9611612
references.citation: N Engl J Med. 2007 May 24;356(21):2211-2; author reply 2213-4
references.pmid: 17526088
references.citation: Respir Med. 2006 Mar;100(3):385-92
references.pmid: 16105731
references.citation: Ann Intern Med. 2007 Apr 17;146(8):545-55
references.pmid: 17310045
references.citation: Cancer Epidemiol Biomarkers Prev. 2011 Feb;20(2):337-44
references.pmid: 21148757
references.citation: Eur Respir J. 2002 Oct;20(4):819-25
references.pmid: 12412670