Evaluation of new treatments in radiation oncology: are they better than standard treatments?
BACKGROUND
The superiority of innovative over standard treatments is not known. To describe accurately the outcomes of innovations that are tested in randomized controlled trials (RCTs) 3 factors have to be considered: publication rate, quality of trials, and the choice of the adequate comparator intervention.


OBJECTIVE
To determine the success rate of innovative treatments by assessing preferences between experimental and standard treatments according to original investigators' conclusions, determining the proportion of RCTs that achieved primary outcomes' statistical significance, and performing meta-analysis to examine if the summary point estimate favored innovative vs standard treatments.


METHODS
Randomized controlled trials conducted by the Radiation Therapy Oncology Group (RTOG).


METHODS
All completed phase 3 trials conducted by the RTOG since its creation in 1968 until 2002. For multiple publications of the same study, we used the one with the most complete primary outcomes and with the longest follow-up information.


METHODS
We used the US National Cancer Institute definition of completed studies to determine the publication rate. We extracted data related to publication status, methodological quality, and treatment comparisons. One investigator extracted the data from all studies and 2 independent investigators extracted randomly about 50% of the data. Disagreements were resolved by consensus during a meeting.


RESULTS
Data on 12,734 patients from 57 trials were evaluated. The publication rate was 95%. The quality of trials was high. We found no evidence of inappropriateness of the choice of comparator. Although the investigators judged that standard treatments were preferred in 71% of the comparisons, when data were meta-analyzed innovations were as likely as standard treatments to be successful (odds ratio for survival, 1.01; 99% confidence interval, 0.96-1.07; P = .5). In contrast, treatment-related mortality was worse with innovations (odds ratio, 1.76; 99% confidence interval, 1.01-3.07; P = .008). We found no predictable pattern of treatment successes in oncology: sometimes innovative treatments are better than the standard ones and vice versa; in most cases there were no substantive differences between experimental and conventional treatments.


CONCLUSIONS
The finding that the results in individual trials cannot be predicted in advance indicates that the system and rationale for RCTs is well preserved and that successful interventions can only be identified after an RCT is completed.
Soares
Heloisa P
HP
H. Lee Moffitt Cancer Center and Research Institute at University of South Florida, Tampa 33612, USA.
Kumar
Ambuj
A
Daniels
Stephanie
S
Swann
Suzanne
S
Cantor
Alan
A
Hozo
Iztok
I
Clark
Mike
M
Serdarevic
Fadila
F
Gwede
Clement
C
Trotti
Andy
A
Djulbegovic
Benjamin
B
D017326:Clinical Trials, Phase III as Topic; D006801:Humans; D009369:Neoplasms / Q000532:radiotherapy*; D017063:Outcome Assessment, Health Care; D018787:Radiation Oncology; D016032:Randomized Controlled Trials as Topic
JAMA. 2001 Apr 18;285(15):1987-91
11308435
Science. 1977 Nov 18;198(4318):684-9
333585
BMJ. 2003 Jun 28;326(7404):1453-5
12829562
BMJ. 1997 Jan 4;314(7073):74-5
9001503
N Engl J Med. 1994 Aug 11;331(6):394-8
8028622
J R Soc Med. 2013 Jan;106(1):30-3
23358277
BMJ. 1994 Jan 29;308(6924):283-4
8124111
BMJ. 2004 Jan 3;328(7430):22-4
14703540
BMJ. 2004 Feb 28;328(7438):476-7
14988165
Stat Med. 1989 Apr;8(4):441-54
2727468
JAMA. 2001 Apr 18;285(15):1992-5
11308436
Biometrics. 1977 Jun;33(2):363-74
884196
BMJ. 2001 Mar 31;322(7289):795
11322146
JAMA. 2001 Jun 6;285(21):2713-4
11386921
Curr Oncol Rep. 2001 Sep;3(5):389-95
11489238
Stat Med. 1998 Dec 30;17(24):2815-34
9921604
N Engl J Med. 2000 Jan 6;342(1):42-9
10620649
Lancet. 2001 Feb 3;357(9253):373-80
11211013
N Engl J Med. 1987 Jul 16;317(3):141-5
3600702
JAMA. 1990 May 9;263(18):2493, 2498, 2501 passim
2329639
BMC Med Res Methodol. 2003 Apr 22;3:7
12709266
BMJ. 2002 Dec 14;325(7377):1420; author reply 1420
12480865
Lancet. 2000 Aug 19;356(9230):635-8
10968436
Ann Intern Med. 2004 Nov 16;141(10):781-8
15545678
Account Res. 2003 Oct-Dec;10(4):301-15
14989285
JAMA. 1992 Jan 15;267(3):374-8
1727960
BMJ. 2004 Feb 28;328(7438):475-6
14988164
Clin Oncol (R Coll Radiol). 1997;9(2):100-14
9135895
JAMA. 2003 Aug 20;290(7):921-8
12928469
BMJ. 1995 Aug 19;311(7003):485
7647644
Health Technol Assess. 1998 Dec;2(15):i-vi, 1-132
10194615
JAMA. 1995 Feb 1;273(5):408-12
7823387
Ann Intern Med. 2001 Apr 17;134(8):663-94
11304107
AIDS Educ Prev. 1997 Feb;9(1 Suppl):15-21
9083596
JAMA. 2001 Mar 7;285(9):1206-8
11231752
J Clin Epidemiol. 1995 Jan;48(1):45-57; discussion 59-60
7853047
BMJ. 1998 Oct 31;317(7167):1170-1
9794846
Int J Technol Assess Health Care. 1988;4(4):637-42
10291102