Believability of relative risks and odds ratios in abstracts: cross sectional study.
OBJECTIVE
To compare the distribution of P values in abstracts of randomised controlled trials with that in observational studies, and to check P values between 0.04 and 0.06.


METHODS
Cross sectional study of all 260 abstracts in PubMed of articles published in 2003 that contained "relative risk" or "odds ratio" and reported results from a randomised trial, and random samples of 130 abstracts from cohort studies and 130 from case-control studies. P values were noted or calculated if unreported.


METHODS
Prevalence of significant P values in abstracts and distribution of P values between 0.04 and 0.06.


RESULTS
The first result in the abstract was statistically significant in 70% of the trials, 84% of cohort studies, and 84% of case-control studies. Although many of these results were derived from subgroup or secondary analyses, or biased selection of results, they were presented without reservations in 98% of the trials. P values were more extreme in observational studies (P < 0.001) and in cohort studies than in case-control studies (P = 0.04). The distribution of P values around P = 0.05 was extremely skewed. Only five trials had 0.05 < or = P < 0.06, whereas 29 trials had 0.04 < or = P < 0.05. I could check the calculations for 27 of these trials. One of four non-significant results was significant. Four of the 23 significant results were wrong, five were doubtful, and four could be discussed. Nine cohort studies and eight case-control studies reported P values between 0.04 and 0.06, but in all 17 cases P < 0.05. Because the analyses had been adjusted for confounders, these results could not be checked.


CONCLUSIONS
Significant results in abstracts are common but should generally be disbelieved.
Gøtzsche
Peter C
PC
Nordic Cochrane Centre, H:S Rigshospitalet, DK-2100 Copenhagen Ø, Denmark,. pcg@cochrane.dk
D000043:Abstracting and Indexing / Q000592:standards* / Q000706:statistics & numerical data; D016022:Case-Control Studies; D015331:Cohort Studies; D003430:Cross-Sectional Studies; D016017:Odds Ratio; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D018570:Risk Assessment