Overinterpretation and misreporting of diagnostic accuracy studies: evidence of "spin".
OBJECTIVE
To estimate the frequency of distorted presentation and overinterpretation of results in diagnostic accuracy studies.


METHODS
MEDLINE was searched for diagnostic accuracy studies published between January and June 2010 in journals with an impact factor of 4 or higher. Articles included were primary studies of the accuracy of one or more tests in which the results were compared with a clinical reference standard. Two authors scored each article independently by using a pretested data-extraction form to identify actual overinterpretation and practices that facilitate overinterpretation, such as incomplete reporting of study methods or the use of inappropriate methods (potential overinterpretation). The frequency of overinterpretation was estimated in all studies and in a subgroup of imaging studies.


RESULTS
Of the 126 articles, 39 (31%; 95% confidence interval [CI]: 23, 39) contained a form of actual overinterpretation, including 29 (23%; 95% CI: 16, 30) with an overly optimistic abstract, 10 (8%; 96% CI: 3%, 13%) with a discrepancy between the study aim and conclusion, and eight with conclusions based on selected subgroups. In our analysis of potential overinterpretation, authors of 89% (95% CI: 83%, 94%) of the studies did not include a sample size calculation, 88% (95% CI: 82%, 94%) did not state a test hypothesis, and 57% (95% CI: 48%, 66%) did not report CIs of accuracy measurements. In 43% (95% CI: 34%, 52%) of studies, authors were unclear about the intended role of the test, and in 3% (95% CI: 0%, 6%) they used inappropriate statistical tests. A subgroup analysis of imaging studies showed 16 (30%; 95% CI: 17%, 43%) and 53 (100%; 95% CI: 92%, 100%) contained forms of actual and potential overinterpretation, respectively.


CONCLUSIONS
Overinterpretation and misreporting of results in diagnostic accuracy studies is frequent in journals with high impact factors.


BACKGROUND
http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.12120527/-/DC1.
Department of Clinical Epidemiology, Academic Medical Centre, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, the Netherlands. e.a.ochodo@amc.uva.nl
Eleanor A
EA
Ochodo
Margriet C
MC
de Haan
Johannes B
JB
Reitsma
Lotty
L
Hooft
Patrick M
PM
Bossuyt
Mariska M G
MM
Leeflang
D016001:Confidence Intervals; D003952:Diagnostic Imaging / Q000592:standards*; D003955:Diagnostic Tests, Routine; D019983:Guideline Adherence; D017408:Guidelines as Topic; D006801:Humans; D055811:Journal Impact Factor; D018156:Journalism, Medical / Q000592:standards*; D010506:Periodicals as Topic / Q000592:standards*; D011643:Publishing / Q000592:standards*; D012107:Research Design / Q000592:standards*