title: Evaluation of new treatments in radiation oncology: are they better than standard treatments?
abstract: BACKGROUND
  The superiority of innovative over standard treatments is not known. To describe accurately the outcomes of innovations that are tested in randomized controlled trials (RCTs) 3 factors have to be considered: publication rate, quality of trials, and the choice of the adequate comparator intervention.
  
  
  OBJECTIVE
  To determine the success rate of innovative treatments by assessing preferences between experimental and standard treatments according to original investigators' conclusions, determining the proportion of RCTs that achieved primary outcomes' statistical significance, and performing meta-analysis to examine if the summary point estimate favored innovative vs standard treatments.
  
  
  METHODS
  Randomized controlled trials conducted by the Radiation Therapy Oncology Group (RTOG).
  
  
  METHODS
  All completed phase 3 trials conducted by the RTOG since its creation in 1968 until 2002. For multiple publications of the same study, we used the one with the most complete primary outcomes and with the longest follow-up information.
  
  
  METHODS
  We used the US National Cancer Institute definition of completed studies to determine the publication rate. We extracted data related to publication status, methodological quality, and treatment comparisons. One investigator extracted the data from all studies and 2 independent investigators extracted randomly about 50% of the data. Disagreements were resolved by consensus during a meeting.
  
  
  RESULTS
  Data on 12,734 patients from 57 trials were evaluated. The publication rate was 95%. The quality of trials was high. We found no evidence of inappropriateness of the choice of comparator. Although the investigators judged that standard treatments were preferred in 71% of the comparisons, when data were meta-analyzed innovations were as likely as standard treatments to be successful (odds ratio for survival, 1.01; 99% confidence interval, 0.96-1.07; P = .5). In contrast, treatment-related mortality was worse with innovations (odds ratio, 1.76; 99% confidence interval, 1.01-3.07; P = .008). We found no predictable pattern of treatment successes in oncology: sometimes innovative treatments are better than the standard ones and vice versa; in most cases there were no substantive differences between experimental and conventional treatments.
  
  
  CONCLUSIONS
  The finding that the results in individual trials cannot be predicted in advance indicates that the system and rationale for RCTs is well preserved and that successful interventions can only be identified after an RCT is completed.
authors.affiliation: H. Lee Moffitt Cancer Center and Research Institute at University of South Florida, Tampa 33612, USA.
authors.forename: Heloisa P
authors.initials: HP
authors.lastname: Soares
authors.forename: Ambuj
authors.initials: A
authors.lastname: Kumar
authors.forename: Stephanie
authors.initials: S
authors.lastname: Daniels
authors.forename: Suzanne
authors.initials: S
authors.lastname: Swann
authors.forename: Alan
authors.initials: A
authors.lastname: Cantor
authors.forename: Iztok
authors.initials: I
authors.lastname: Hozo
authors.forename: Mike
authors.initials: M
authors.lastname: Clark
authors.forename: Fadila
authors.initials: F
authors.lastname: Serdarevic
authors.forename: Clement
authors.initials: C
authors.lastname: Gwede
authors.forename: Andy
authors.initials: A
authors.lastname: Trotti
authors.forename: Benjamin
authors.initials: B
authors.lastname: Djulbegovic
mesh_terms: D017326:Clinical Trials, Phase III as Topic; D006801:Humans; D009369:Neoplasms / Q000532:radiotherapy*; D017063:Outcome Assessment, Health Care; D018787:Radiation Oncology; D016032:Randomized Controlled Trials as Topic
references.citation: JAMA. 2001 Apr 18;285(15):1987-91
references.pmid: 11308435
references.citation: Science. 1977 Nov 18;198(4318):684-9
references.pmid: 333585
references.citation: BMJ. 2003 Jun 28;326(7404):1453-5
references.pmid: 12829562
references.citation: BMJ. 1997 Jan 4;314(7073):74-5
references.pmid: 9001503
references.citation: N Engl J Med. 1994 Aug 11;331(6):394-8
references.pmid: 8028622
references.citation: J R Soc Med. 2013 Jan;106(1):30-3
references.pmid: 23358277
references.citation: BMJ. 1994 Jan 29;308(6924):283-4
references.pmid: 8124111
references.citation: BMJ. 2004 Jan 3;328(7430):22-4
references.pmid: 14703540
references.citation: BMJ. 2004 Feb 28;328(7438):476-7
references.pmid: 14988165
references.citation: Stat Med. 1989 Apr;8(4):441-54
references.pmid: 2727468
references.citation: JAMA. 2001 Apr 18;285(15):1992-5
references.pmid: 11308436
references.citation: Biometrics. 1977 Jun;33(2):363-74
references.pmid: 884196
references.citation: BMJ. 2001 Mar 31;322(7289):795
references.pmid: 11322146
references.citation: JAMA. 2001 Jun 6;285(21):2713-4
references.pmid: 11386921
references.citation: Curr Oncol Rep. 2001 Sep;3(5):389-95
references.pmid: 11489238
references.citation: Stat Med. 1998 Dec 30;17(24):2815-34
references.pmid: 9921604
references.citation: N Engl J Med. 2000 Jan 6;342(1):42-9
references.pmid: 10620649
references.citation: Lancet. 2001 Feb 3;357(9253):373-80
references.pmid: 11211013
references.citation: N Engl J Med. 1987 Jul 16;317(3):141-5
references.pmid: 3600702
references.citation: JAMA. 1990 May 9;263(18):2493, 2498, 2501 passim
references.pmid: 2329639
references.citation: BMC Med Res Methodol. 2003 Apr 22;3:7
references.pmid: 12709266
references.citation: BMJ. 2002 Dec 14;325(7377):1420; author reply 1420
references.pmid: 12480865
references.citation: Lancet. 2000 Aug 19;356(9230):635-8
references.pmid: 10968436
references.citation: Ann Intern Med. 2004 Nov 16;141(10):781-8
references.pmid: 15545678
references.citation: Account Res. 2003 Oct-Dec;10(4):301-15
references.pmid: 14989285
references.citation: JAMA. 1992 Jan 15;267(3):374-8
references.pmid: 1727960
references.citation: BMJ. 2004 Feb 28;328(7438):475-6
references.pmid: 14988164
references.citation: Clin Oncol (R Coll Radiol). 1997;9(2):100-14
references.pmid: 9135895
references.citation: JAMA. 2003 Aug 20;290(7):921-8
references.pmid: 12928469
references.citation: BMJ. 1995 Aug 19;311(7003):485
references.pmid: 7647644
references.citation: Health Technol Assess. 1998 Dec;2(15):i-vi, 1-132
references.pmid: 10194615
references.citation: JAMA. 1995 Feb 1;273(5):408-12
references.pmid: 7823387
references.citation: Ann Intern Med. 2001 Apr 17;134(8):663-94
references.pmid: 11304107
references.citation: AIDS Educ Prev. 1997 Feb;9(1 Suppl):15-21
references.pmid: 9083596
references.citation: JAMA. 2001 Mar 7;285(9):1206-8
references.pmid: 11231752
references.citation: J Clin Epidemiol. 1995 Jan;48(1):45-57; discussion 59-60
references.pmid: 7853047
references.citation: BMJ. 1998 Oct 31;317(7167):1170-1
references.pmid: 9794846
references.citation: Int J Technol Assess Health Care. 1988;4(4):637-42
references.pmid: 10291102