Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors.
BACKGROUND
We wanted to evaluate the impact of nonblinded outcome assessors on estimated treatment effects in time-to-event trials.


METHODS
Systematic review of randomized clinical trials with both blinded and nonblinded assessors of the same time-to-event outcome. Two authors agreed on inclusion of trials and outcomes. We compared hazard ratios based on nonblinded and blinded assessments. A ratio of hazard ratios (RHR)<1 indicated that nonblinded assessors generated more optimistic effect estimates. We pooled RHRs with inverse variance random-effects meta-analysis.


RESULTS
We included 18 trials. Eleven trials (1969 patients) with subjective outcomes provided hazard ratios, RHR 0.88 (0.69 to 1.12), (I2=44%, P=0.06), but unconditional pooling was problematic because of qualitative heterogeneity. Four atypical cytomegalovirus retinitis trials compared experimental oral administration with control intravenous administration of the same drug, resulting in bias favouring the control intervention, RHR 1.33 (0.98 to 1.82). Seven trials of cytomegalovirus retinitis, tibial fracture and multiple sclerosis compared experimental interventions with standard control interventions, e.g. placebo, no-treatment or active control, resulting in bias favouring the experimental intervention, RHR 0.73 (0.57 to 0.93), indicating an average exaggeration of nonblinded hazard ratios by 27% (7% to 43%).


CONCLUSIONS
Lack of blinded outcome assessors in randomized trials with subjective time-to-event outcomes causes high risk of observer bias. Nonblinded outcome assessors typically favour the experimental intervention, exaggerating the hazard ratio by an average of approximately 27%; but in special situations, nonblinded outcome assessors favour control interventions, inducing a comparable degree of observer bias in the reversed direction.
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France ah@cochrane.dk.
Asbjørn
A
Hróbjartsson
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Ann Sofia Skou
AS
Thomsen
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Frida
F
Emanuelsson
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Britta
B
Tendal
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Jeppe Vejlgaard
JV
Rasmussen
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Jørgen
J
Hilden
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Isabelle
I
Boutron
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Philippe
P
Ravaud
Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark, Department of Orthopaedic Surgery, Herlev University Hospital, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and French Cochrane Centre, Assistance Publique (Hotel Dieu), Université Paris Descartes, Paris, France.
Stig
S
Brorson
D006801:Humans; D015588:Observer Variation; D016032:Randomized Controlled Trials as Topic / Q000379:methods*; D012107:Research Design; D016037:Single-Blind Method
Randomized clinical trials; bias; blinding; observer bias; time-to-event