title: Including non-public data and studies in systematic reviews and systematic maps.
abstract: Systematic reviews and maps should be based on the best available evidence, and reviewers should make all reasonable efforts to source and include potentially relevant studies. However, reviewers may not be able to consider all existing evidence, since some data and studies may not be publicly available. Including non-public studies in reviews provides a valuable opportunity to increase systematic review/map comprehensiveness, potentially mitigating negative impacts of publication bias. Studies may be non-public for many reasons: some may still be in the process of being published (publication can take a long time); some may not be published due to author/publisher restrictions; publication bias may make it difficult to publish non-significant or negative results. Here, we consider what forms these non-public studies may take and the implications of including them in systematic reviews and maps. Reviewers should carefully consider the advantages and disadvantages of including non-public studies, weighing risks of bias against benefits of increased comprehensiveness. As with all systematic reviews and maps, reviewers must be transparent about methods used to obtain data and avoid risks of bias in their synthesis. We make tentative suggestions for reviewers in situations where non-public data may be present in an evidence base.
authors.affiliation: MISTRA EviEM, Stockholm Environment Institute, Box 24218, 10451 Stockholm, Sweden. Electronic address: neal_haddaway@hotmail.com.
authors.forename: Neal R
authors.initials: NR
authors.lastname: Haddaway
authors.affiliation: Centre for Environmental Policy, Imperial College, London, UK. Electronic address: alexandra.collins@imperial.ac.uk.
authors.forename: Alexandra M
authors.initials: AM
authors.lastname: Collins
authors.affiliation: Department for Civil and Environmental Engineering, Imperial College, London, UK. Electronic address: drdcoughlin@googlemail.com.
authors.forename: Deborah
authors.initials: D
authors.lastname: Coughlin
authors.affiliation: Institute for Biosafety in Plant Biotechnology, Julius Kühn-Institute (JKI), Federal Research Centre for Cultivated Plants, Erwin Baur Str. 27, 06484 Quedlinburg, Germany. Electronic address: christian.kohl@julius-kuehn.de.
authors.forename: Christian
authors.initials: C
authors.lastname: Kohl
mesh_terms: D022126:Access to Information; D004782:Environmental Health / Q000592:standards; D006801:Humans; D008377:Maps as Topic; D011643:Publishing / Q000592:standards; D012196:Review Literature as Topic
keywords: Commercial data; Dark data; Grey literature; Industrial data; Private data; Restricted access