Inadvertent P-hacking among trials and systematic reviews of the effect of progestogens in pregnancy? A systematic review and meta-analysis.
BACKGROUND
Progestogens have been evaluated in numerous trials and meta-analyses, many of which concluded they were effective. However, two large trials PROMISE and OPPTIMUM have recently concluded that progesterone was ineffective. This raises the possibility that earlier studies and reviews had been biased by either selective publication or selective choice of outcomes, so called "P-hacking".


OBJECTIVE
To compare the findings all progestogen trials and systematic reviews with those of trials with pre-registered primary outcomes which avoided selective outcome reporting.


METHODS
Search of PubMed, the Cochrane Library and trial registries. Registration PROSPERO CRD42016035303.


METHODS
Systematic reviews of randomised trials comparing progestogen with placebo in pregnancy and the individual trials included in those reviews. The subset of trials reporting a pre-registered primary outcome were compared with the totality of trials and reviews.


METHODS
For reviews all outcomes were included. For individual trials all outcomes reported in the systematic reviews were included. For the comparison group we recorded the registered primary outcome from trials that were either registered before they started, or registered during the recruitment phase and also double blind.


RESULTS
Nineteen of twenty-nine meta-analyses concluded that progestogens were effective. Twenty-two trials reported their pre-registered primary outcomes. There was no effect of progesterone on primary registered dichotomous outcome RR 1.00 (95% CI 0.94-1.07). Only one of the 22 showed a nominally statistically significant benefit.


CONCLUSIONS
When evaluated in registered double-blind trials with analysis restricted to predefined primary outcomes, progestational agents in pregnancy are ineffective.


CONCLUSIONS
Progestogens to prevent pregnancy loss, an example of P-hacking.
Department of Child Health Obstetrics and Gynaecology, Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, UK.
M
M
Prior
Department of Child Health Obstetrics and Gynaecology, Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, UK.
R
R
Hibberd
Department of Child Health Obstetrics and Gynaecology, Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, UK.
N
N
Asemota
Department of Child Health Obstetrics and Gynaecology, Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, UK.
J G
JG
Thornton
D005260:Female; D006801:Humans; D011247:Pregnancy; D015982:Bias; D002986:Clinical Trials as Topic; D011248:Pregnancy Complications / Q000453:epidemiology / Q000517:prevention & control; D011256:Pregnancy Outcome / Q000453:epidemiology; D011374:Progesterone / Q000627:therapeutic use; D012107:Research Design; D000078202:Systematic Reviews as Topic
D011374:Progesterone
Miscarriage; P-hacking; outcome switching; pregnancy loss; preterm birth; progestogen