Assessing the potential for outcome reporting bias in a review: a tutorial.
BACKGROUND
Outcome reporting bias (ORB) occurs when variables are selected for publication based on their results. This can impact upon the results of a meta-analysis, biasing the pooled treatment effect estimate.The aim of this paper is to show how to assess a systematic review and corresponding trial reports for ORB using an example review of intravenous and nebulised magnesium in the treatment of asthma.


METHODS
The review was assessed for ORB by 1) checking the reasons, when available, for excluding studies to ensure that no studies were excluded because they did not report the outcomes of interest in the review; 2) assessing the eligible studies as to whether the review outcomes of interest were reported. Each study was classified using a system developed in the ORBIT (Outcome Reporting Bias In Trials) project to indicate whether ORB was suspected and a reason for the suspicion. Authors of trials that did not report the outcomes of interest were contacted for information. A sensitivity analysis was performed to assess the robustness of the conclusions of the review to this potential source of bias.


RESULTS
Twenty-four studies were included in the review; two studies had been excluded for not reporting either of the two outcomes of interest. Six included studies did not report hospital admission and two did not report pulmonary function. There was high suspicion of outcome reporting bias in four studies. Results from the sensitivity analysis indicate that review conclusions were not overturned.


CONCLUSIONS
This paper demonstrates, with the example of the magnesium review, how to assess a review for outcome reporting bias. A review should not exclude studies if they have not reported the outcomes of interest and should consider the potential for outcome reporting bias in all included studies.
Dwan
Kerry
K
Centre for Medical Statistics and Health Evaluation, University of Liverpool, UK. kerry.dwan@liverpool.ac.uk
Gamble
Carrol
C
Kolamunnage-Dona
Ruwanthi
R
Mohammed
Shabana
S
Powell
Colin
C
Williamson
Paula R
PR
D000280:Administration, Inhalation; D000328:Adult; D018927:Anti-Asthmatic Agents / Q000008:administration & dosage; D001249:Asthma / Q000188:drug therapy / Q000503:physiopathology; D002648:Child; D002986:Clinical Trials as Topic; D023381:Endpoint Determination; D019317:Evidence-Based Medicine; D006801:Humans; D008274:Magnesium / Q000008:administration & dosage; D015201:Meta-Analysis as Topic; D009330:Nebulizers and Vaporizers; D017594:Publication Bias; D015203:Reproducibility of Results; D012196:Review Literature as Topic; D016896:Treatment Outcome
D018927:Anti-Asthmatic Agents; D008274:Magnesium
Chest. 1993 Sep;104(3):831-4
8365297
PLoS Med. 2007 Mar 27;4(3):e78
17388659
Cochrane Database Syst Rev. 2009 Jan 21;(1):MR000006
19160345
Stat Methods Med Res. 2005 Oct;14(5):515-24
16248351
J Pediatr (Rio J). 2001 Jul-Aug;77(4):279-87
14647859
Indian Pediatr. 1997 May;34(5):389-97
9332112
J Pediatr. 1996 Dec;129(6):809-14
8969721
BMJ. 2010 Feb 15;340:c365
20156912
Stat Med. 2005 May 30;24(10):1547-61
15580591
Ann Emerg Med. 2002 Jun;39(6):585-91
12023699
BMJ. 2003 Sep 6;327(7414):557-60
12958120
J Clin Epidemiol. 2007 Apr;60(4):321-3
17346603
Arch Pediatr Adolesc Med. 2000 Oct;154(10):979-83
11030848
Pediatr Emerg Care. 2004 Jan;20(1):22-6
14716161
Eur J Emerg Med. 2001 Mar;8(1):9-15
11314826
BMJ. 2008 Jun 21;336(7658):1413-5
18566080
Ann Emerg Med. 2000 Dec;36(6):572-8
11097697
Turk J Pediatr. 1996 Apr-Jun;38(2):169-75
8701480
Trials. 2007 Mar 06;8:9
17341316
Respirology. 2000 Sep;5(3):221-5
11022983
Emerg Med J. 2007 Dec;24(12):823-30
18029512
PLoS One. 2008 Aug 28;3(8):e3081
18769481
Eur J Emerg Med. 1999 Sep;6(3):201-5
10622383
PLoS Med. 2008 Apr 29;5(4):e96
18447577
JAMA. 2004 May 26;291(20):2457-65
15161896