Reporting bias in medical research - a narrative review.
Reporting bias represents a major problem in the assessment of health care interventions. Several prominent cases have been described in the literature, for example, in the reporting of trials of antidepressants, Class I anti-arrhythmic drugs, and selective COX-2 inhibitors. The aim of this narrative review is to gain an overview of reporting bias in the medical literature, focussing on publication bias and selective outcome reporting. We explore whether these types of bias have been shown in areas beyond the well-known cases noted above, in order to gain an impression of how widespread the problem is. For this purpose, we screened relevant articles on reporting bias that had previously been obtained by the German Institute for Quality and Efficiency in Health Care in the context of its health technology assessment reports and other research work, together with the reference lists of these articles.We identified reporting bias in 40 indications comprising around 50 different pharmacological, surgical (e.g. vacuum-assisted closure therapy), diagnostic (e.g. ultrasound), and preventive (e.g. cancer vaccines) interventions. Regarding pharmacological interventions, cases of reporting bias were, for example, identified in the treatment of the following conditions: depression, bipolar disorder, schizophrenia, anxiety disorder, attention-deficit hyperactivity disorder, Alzheimer's disease, pain, migraine, cardiovascular disease, gastric ulcers, irritable bowel syndrome, urinary incontinence, atopic dermatitis, diabetes mellitus type 2, hypercholesterolaemia, thyroid disorders, menopausal symptoms, various types of cancer (e.g. ovarian cancer and melanoma), various types of infections (e.g. HIV, influenza and Hepatitis B), and acute trauma. Many cases involved the withholding of study data by manufacturers and regulatory agencies or the active attempt by manufacturers to suppress publication. The ascertained effects of reporting bias included the overestimation of efficacy and the underestimation of safety risks of interventions.In conclusion, reporting bias is a widespread phenomenon in the medical literature. Mandatory prospective registration of trials and public access to study data via results databases need to be introduced on a worldwide scale. This will allow for an independent review of research data, help fulfil ethical obligations towards patients, and ensure a basis for fully-informed decision making in the health care system.
McGauran
Natalie
N
Institute for Quality and Efficiency in Health Care, Dillenburger Str 27, 51105 Cologne, Germany. n.mcgauran@iqwig.de
Wieseler
Beate
B
Kreis
Julia
J
Schüler
Yvonne-Beatrice
YB
Kölsch
Heike
H
Kaiser
Thomas
T
D022126:Access to Information; D000818:Animals; D035843:Biomedical Research; D003257:Consumer Product Safety; D019992:Databases as Topic; D019317:Evidence-Based Medicine; D006801:Humans; D010506:Periodicals as Topic; D017594:Publication Bias; D012042:Registries; D015203:Reproducibility of Results; D016896:Treatment Outcome; D014342:Truth Disclosure
Can J Psychiatry. 2004 Sep;49(9):601-6
15503731
CMAJ. 2004 Sep 28;171(7):735-40
15451835
CMAJ. 2004 Feb 17;170(4):477-80
14970094
BMJ. 2008 Mar 8;336(7643):532-4
18304990
Swiss Med Wkly. 2008 Apr 5;138(13-14):197-203
18389392
J Clin Oncol. 1986 Oct;4(10):1529-41
3760920
Arch Intern Med. 1989 Dec;149(12):2694-8
2688587
N Engl J Med. 1995 Oct 26;333(17):1099-105
7565947
BMJ. 2009 Dec 08;339:b5106
19995812
Eur J Cancer. 2007 Nov;43(17):2559-79
17981458
BMJ. 2007 Nov 17;335(7628):1015
18006992
N Engl J Med. 2008 Apr 3;358(14):1431-43
18376000
Diabetes Care. 2008 Mar;31(3):e10
18308673
JAMA. 1997 Apr 16;277(15):1238-43
9103350
Br J Cancer. 2003 Oct 20;89(8):1405-8
14562007
BMJ. 1998 Jul 25;317(7153):235-40
9677209
N Engl J Med. 2008 Jan 17;358(3):252-60
18199864
BMJ. 2005 Apr 23;330(7497):956-8
15845980
J Natl Cancer Inst. 2005 Jul 20;97(14):1043-55
16030302
Online J Curr Clin Trials. 1993 Apr 28;Doc No 50:[4967 words; 53 paragraphs]
8306005
Curr Control Trials Cardiovasc Med. 2001;2(5):205-207
11806796
BMJ. 2007 Jan 20;334(7585):120-3
17235089
MMW Fortschr Med. 2006 Oct 26;148(43):8
17619412
Br Med J. 1980 Mar 22;280(6217):833-6
7370687
J Gen Intern Med. 1998 Sep;13(9):600-6
9754515
Control Clin Trials. 1997 Oct;18(5):431-44
9315426
BMJ. 1995 May 20;310(6990):1313-5
7773049
Cochrane Database Syst Rev. 2009 Jan 21;(1):MR000006
19160345
JAMA. 2008 Apr 16;299(15):1833-5
18413880
BMJ. 2001 Aug 25;323(7310):415
11520825
Lancet. 2008 Jul 19;372(9634):201
18640445
Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000010
17443631
N Engl J Med. 2005 Dec 29;353(26):2813-4
16339408
JAMA. 1998 Jul 15;280(3):254-7
9676673
JAMA. 2003 Jul 23;290(4):495-501
12876092
Br J Psychiatry. 2003 Dec;183:498-506
14645020
BMC Med Res Methodol. 2008 Feb 11;8:4
18267008
Lancet. 2003 Mar 22;361(9362):1015-6
12660062
JAMA. 2003 Nov 19;290(19):2534
14625316
N Engl J Med. 2005 Apr 28;352(17):1739-41
15858180
JAMA. 1990 Mar 9;263(10):1405-8
2304220
BMJ. 2002 Aug 3;325(7358):249
12153921
J Clin Epidemiol. 2002 Apr;55(4):407-10
11927210
N Engl J Med. 2007 Jun 28;356(26):2734-6
17548427
N Engl J Med. 2006 Nov 23;355(21):2171-3
17124013
PLoS Med. 2004 Dec;1(3):e60
15562322
N Engl J Med. 2007 Aug 30;357(9):844-6
17687124
N Engl J Med. 2009 Jan 8;360(2):103-6
19129523
J Natl Cancer Inst. 1989 Jan 18;81(2):107-15
2642556
JAMA. 2003 Jan 22-29;289(4):454-65
12533125
BMJ. 2003 Mar 8;326(7388):518
12623898
J Manag Care Pharm. 2003 Nov-Dec;9(6):559-68
14664664
N Engl J Med. 2007 Jul 5;357(1):28-38
17551159
JAMA. 2001 Aug 22-29;286(8):954-9
11509060
Am J Psychiatry. 2006 Feb;163(2):185-94
16449469
BMJ. 2009 Jun 22;338:b2521
19546133
BMJ. 2004 Jun 26;328(7455):1513
15217848
Br Med J (Clin Res Ed). 1987 Sep 12;295(6599):654-6
3117277
JAMA. 1988 Oct 7;260(13):1910-6
3047448
J Bone Joint Surg Am. 2008 Mar;90(3):595-601
18310710
JAMA. 2007 Sep 12;298(10):1189-95
17848653
Croat Med J. 2008 Apr;49(2):267-8
18461682
BMJ. 2008 Jan 26;336(7637):170
18219000
Nature. 2003 Jun 19;423(6942):792
12815392
BMJ. 2003 May 31;326(7400):1171-3
12775615
BMC Res Notes. 2009 Jan 10;2:5
19134216
Cancer. 1987 Dec 1;60(11):2811-5
3315175
Chest. 2009 Jul;136(1):295-303
19584212
Br J Psychiatry. 2003 Aug;183:161-6
12893670
J Clin Epidemiol. 2003 Jan;56(1):44-51
12589869
CMAJ. 2002 Jun 25;166(13):1649-50
12126315
JAMA. 2008 Apr 16;299(15):1800-12
18413874
JAMA. 1992 Jan 15;267(3):374-8
1727960
PLoS Med. 2008 Sep 23;5(9):e191
18816163
J Med Ethics. 2008 Sep;34(9):e20
18757621
Lancet. 2000 Mar 25;355(9209):1035-40
10744088
J Am Acad Dermatol. 2005 Jun;52(6):967-71
15928613
N Engl J Med. 2009 Feb 19;360(8):824-30
19228628
JAMA. 2003 Aug 20;290(7):921-8
12928469
JAMA. 1998 Jul 15;280(3):250-3
9676672
N Engl J Med. 2007 Jul 5;357(1):63-4
17551160
JAMA. 2006 May 17;295(19):2270-4
16705108
JAMA. 2005 Apr 20;293(15):1900-5
15840865
N Engl J Med. 2004 Oct 21;351(17):1707-9
15470193
Medscape J Med. 2008;10(9):211
19008973
BMJ. 2005 Apr 2;330(7494):753
15681569
BMJ. 1997 Sep 13;315(7109):635-40
9310564
N Engl J Med. 2007 Jun 14;356(24):2457-71
17517853
BMJ. 2009 Jun 25;338:b2473
19556275
N Engl J Med. 1996 Feb 8;334(6):392-4
8538713
Lancet. 2004 Apr 24;363(9418):1341-5
15110490
Isr Med Assoc J. 2006 Jul;8(7):451-4
16889157
JAMA. 2002 Jan 23-30;287(4):473-80
11798369
N Engl J Med. 2007 Jun 14;356(24):2522-4
17517854
Lancet. 2006 Dec 9;368(9552):2049; author reply 2050-1
17161720
Psychol Med. 2006 Nov;36(11):1647-56
16893480
N Engl J Med. 2007 Apr 19;356(16):1601-4
17442902
Med J Aust. 2006 Jun 19;184(12):621-6
16803442
Lancet. 1999 Mar 6;353(9155):771-2
10459956
BMC Med Res Methodol. 2002 Apr 26;2:7
11978183
Ann Intern Med. 2008 Aug 19;149(4):251-8
18711155
Cochrane Database Syst Rev. 2006 Jul 19;(3):CD001265
16855962
Drug Saf. 2006;29(5):385-96
16689555
JAMA. 2001 Jan 24-31;285(4):437-43
11242428
N Engl J Med. 2005 Jun 9;352(23):2436-8
15911780
Lancet. 1985 Oct 12;2(8459):793-5
2864530
Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000005
17443628
BMJ. 2008 May 31;336(7655):1208-9
18511780
CMAJ. 2004 Feb 17;170(4):487-9
14970096
JAMA. 2004 Dec 1;292(21):2622-31
15572720
BMJ. 2003 Dec 13;327(7428):1358-9
14670851
CMAJ. 2001 Sep 4;165(5):632
11563216
Lancet. 2002 Oct 5;360(9339):1071-3
12383990
JAMA. 2004 Dec 1;292(21):2655-7; discussion 2658-9
15572725
Cochrane Database Syst Rev. 2003;(2):CD003592
12804478
Oncologist. 2008 Sep;13(9):930-2
18794215
Health Aff (Millwood). 2007 Mar-Apr;26(2):483-91
17339677
Control Clin Trials. 1987 Dec;8(4):343-53
3442991
N Engl J Med. 2004 Sep 16;351(12):1250-1
15356289
JAMA. 2008 Feb 27;299(8):885-7
18314425
BMC Clin Pharmacol. 2001;1:3
11591227
PLoS Med. 2005 Nov;2(11):e378
16221000
BMJ. 2004 Jun 12;328(7453):1395
15191963
CMAJ. 2004 Feb 17;170(4):489-91
14970097
BMJ. 2002 Sep 14;325(7364):592-5
12228140
N Engl J Med. 2006 Jan 26;354(4):353-65
16436767
Oncologist. 2008 Sep;13(9):925-9
18794216
JAMA. 1998 Jan 28;279(4):281-6
9450711
Transfusion. 2006 Mar;46(3):327-38
16533273
JAMA. 2008 Feb 27;299(8):953-5
18314439
N Engl J Med. 2009 Nov 12;361(20):1963-71
19907043
Lancet. 1997 Aug 2;350(9074):326-9
9251637
Ann Intern Med. 2006 Aug 15;145(4):284-93
16908919
BMJ. 2003 May 31;326(7400):1167-70
12775614
PLoS Med. 2007 Apr;4(4):e134
17455990
BMJ. 1997 Apr 19;314(7088):1145
9146378
Chest. 2009 Jul;136(1):304-305
19584213
N Engl J Med. 2000 Nov 23;343(21):1520-8, 2 p following 1528
11087881
JAMA. 2008 Apr 16;299(15):1813-7
18413875
Science. 2008 Oct 3;322(5898):44-6
18832629
JAMA. 1999 Nov 10;282(18):1752-9
10568648
BMJ. 2002 Sep 7;325(7363):528-31
12217994
BMJ. 2009 May 11;338:b1914
19433504
PLoS One. 2008 Aug 28;3(8):e3081
18769481
BMJ. 2005 Jul 2;331(7507):19
15967761
Am J Psychiatry. 2009 Feb;166(2):140-5
19188290
N Engl J Med. 1989 Aug 10;321(6):406-12
2473403
Contemp Clin Trials. 2008 Mar;29(2):109-13
17919992
JAMA. 1998 May 20;279(19):1566-70
9605902
Circulation. 2005 Sep 27;112(13):2040-2
16172264
Control Clin Trials. 2004 Dec;25(6):598-612
15588746
JAMA. 1997 Apr 16;277(15):1205-13
9103344
Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000011
17443632
Am J Cardiol. 1983 Sep 22;52(6):32C-36C
6353896
PLoS Med. 2008 Nov 25;5(11):e217; discussion e217
19067477
Pharmacoepidemiol Drug Saf. 2005 Feb;14(2):101-6
15386705
BMJ. 2005 Jul 9;331(7508):70
16002878
Lancet. 2009 Jun 20;373(9681):2125-35
19501900
Br J Clin Pharmacol. 1998 Aug;46(2):147-50
9723823
Lancet. 1991 Apr 13;337(8746):867-72
1672966
Int J Cardiol. 1993 Jul 1;40(2):161-6
8349379
Am J Psychiatry. 2005 Oct;162(10):1957-60
16199844
Lancet. 1987 May 9;1(8541):1091
2883426
BMJ. 1997 Sep 13;315(7109):640-5
9310565
JAMA. 2002 Jun 5;287(21):2825-8
12038924
N Engl J Med. 2007 Jul 5;357(1):67-9
17551162
BMJ. 2004 Oct 9;329(7470):816
15472245
JAMA. 2000 Sep 13;284(10):1247-55
10979111
Health Technol Assess. 2000;4(10):1-115
10932019
Lancet. 2004 Dec 4-10;364(9450):2021-9
15582059
CNS Drugs. 2004;18(6):355-64; discussion 365-6
15089103
Control Clin Trials. 1988 Mar;9(1):76-81
3356154
Lancet Oncol. 2009 Feb;10(2):135-46
19167925
BMJ. 2008 Jan 26;336(7637):180-1
18219028
BMJ. 2006 Oct 14;333(7572):782
17028106
J Clin Epidemiol. 1995 May;48(5):713-9
7730926
JAMA. 2004 May 26;291(20):2457-65
15161896
Arch Intern Med. 2003 Jul 28;163(14):1667-72
12885681
JAMA. 2006 Sep 13;296(10):1266-73
16968851
JAMA. 1997 May 28;277(20):1597-604
9168289
BMJ. 2006 Jul 8;333(7558):92-5
16825234
Epidemiology. 2005 Mar;16(2):155-63
15703529
Med J Aust. 2005 Jun 6;182(11):557-60
15938681
Lancet. 2000 Dec 9;356(9246):2009-10
11130544
Bull World Health Organ. 2006 Jan;84(1):10-1
16501708
JAMA. 2006 Apr 12;295(14):1645-6
16609085
Science. 1996 Jul 26;273(5274):411
8677431
N Engl J Med. 2007 Jul 5;357(1):64-6
17551161
Lancet. 2000 Oct 7;356(9237):1228-31
11072941
N Engl J Med. 2005 Jun 23;352(25):2576-8
15972862
Health Technol Assess. 2008 Oct;12(32):iii, ix-x, 1-46
18831948
Cancer. 2007 Apr 1;109(7):1239-46
17326054
BMJ. 2009 Dec 08;339:b5351
19995815
PLoS Med. 2008 Feb;5(2):e45
18303940
JAMA. 2003 Jul 23;290(4):516-23
12876095
Lancet. 2001 May 19;357(9268):1544-5
11377636
Science. 1997 Feb 7;275(5301):777-80
9036541