Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study.
BACKGROUND
A high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines.


OBJECTIVE
To examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy.


METHODS
We identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility.


RESULTS
272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy.


CONCLUSIONS
Despite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.
Bio-Medical Library, University of Minnesota, Minneapolis, Minnesota, United States of America.
Jonathan B
http://orcid.org/0000-0003-1723-5087
JB
Koffel
Spencer S. Eccles Health Sciences Library, University of Utah, Salt Lake City, Utah, United States of America.
Melissa L
ML
Rethlefsen
BMJ. 2005 Jan 8;330(7482):68
15619601
PLoS One. 2014 May 01;9(5):e96407
24788774
BMJ Open. 2015 Dec 07;5(12):e008066
26644119
J Psychosom Res. 2016 May;84:84-92
27095164
J Bone Joint Surg Am. 2013 Jun 5;95(11):e771-7
23780547
J Neurol Neurosurg Psychiatry. 2013 Apr;84(4):420-6
22952326
Acad Med. 2011 Aug;86(8):1049-54
21694568
J Clin Epidemiol. 2009 Oct;62(10):1006-12
19631508
PLoS One. 2015 May 04;10(5):e0125931
25938454
PLoS One. 2014 Jan 21;9(1):e85908
24465781
PLoS One. 2013 Dec 26;8(12):e83138
24386151
Ann Surg. 2015 Apr;261(4):685-94
25575252
PLoS One. 2015 Aug 28;10(8):e0136540
26317406
Nurs Outlook. 2015 Jul-Aug;63(4):446-455.e4
26187084
J Clin Epidemiol. 2008 May;61(5):440-8
18394536
JAMA. 2000 Apr 19;283(15):2008-12
10789670
Angle Orthod. 2013 Jan;83(1):158-63
22720835
Eur J Orthod. 2014 Feb;36(1):74-85
23493385
PLoS Med. 2009 Jul 21;6(7):e1000100
19621070
PLoS One. 2013;8(1):e53536
23341949
Res Synth Methods. 2014 Jun;5(2):116-30
26052651
PLoS Med. 2016 May 24;13(5):e1002028
27218655
Radiology. 2013 Nov;269(2):413-26
23824992
J Clin Epidemiol. 2008 Aug;61(8):748-54
18586178
BMJ. 2014 Jun 25;348:g3804
24965222
Br J Surg. 2012 Sep;99(9):1185-94
22777875
J Clin Epidemiol. 2014 Jul;67(7):754-9
24709031
BMJ. 2014 Jan 08;348:f7668
24401468
Phys Ther. 2013 Nov;93(11):1456-66
23744458
Knee Surg Sports Traumatol Arthrosc. 2015 Sep;23(9):2583-9
25037984
J Clin Epidemiol. 2015 Jun;68(6):617-26
25766056
J Med Libr Assoc. 2009 Jan;97(1):21-9
19158999
PLoS One. 2014 Mar 26;9(3):e92508
24671099
J Clin Epidemiol. 2006 Oct;59(10):1057-63
16980145
J Neurosurg. 2014 Apr;120(4):796-810
24460488