title: Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews.
abstract: In scoping reviews, boundaries of relevant evidence may be initially fuzzy, with refined conceptual understanding of interventions and their proposed mechanisms of action an intended output of the scoping process rather than its starting point. Electronic searches are therefore sensitive, often retrieving very large record sets that are impractical to screen in their entirety. This paper describes methods for applying and evaluating the use of text mining (TM) technologies to reduce impractical screening workload in reviews, using examples of two extremely large-scale scoping reviews of public health evidence (choice architecture (CA) and economic environment (EE)). Electronic searches retrieved >800,000 (CA) and >1 million (EE) records. TM technologies were used to prioritise records for manual screening. TM performance was measured prospectively. TM reduced manual screening workload by 90% (CA) and 88% (EE) compared with conventional screening (absolute reductions of ≈430 000 (CA) and ≈378 000 (EE) records). This study expands an emerging corpus of empirical evidence for the use of TM to expedite study selection in reviews. By reducing screening workload to manageable levels, TM made it possible to assemble and configure large, complex evidence bases that crossed research discipline boundaries. These methods are transferable to other scoping and systematic reviews incorporating conceptual development or explanatory dimensions.
authors.affiliation: Behaviour and Health Research Unit, University of Cambridge, Cambridge, UK.
authors.forename: Ian
authors.initials: I
authors.lastname: Shemilt
authors.affiliation: Thomas Coram Research Unit, Department of Children and Health, Institute of Education, London, UK.
authors.forename: Antonia
authors.initials: A
authors.lastname: Simon
authors.affiliation: Behaviour and Health Research Unit, University of Cambridge, Cambridge, UK.
authors.forename: Gareth J
authors.initials: GJ
authors.lastname: Hollands
authors.affiliation: Behaviour and Health Research Unit, University of Cambridge, Cambridge, UK.
authors.forename: Theresa M
authors.initials: TM
authors.lastname: Marteau
authors.affiliation: Behaviour and Health Research Unit, University of Cambridge, Cambridge, UK.
authors.forename: David
authors.initials: D
authors.lastname: Ogilvie
authors.affiliation: Evidence for Policy and Practice Information and Co-ordinating Centre, Department of Children and Health, Institute of Education, London, UK.
authors.forename: Alison
authors.initials: A
authors.lastname: O'Mara-Eves
authors.affiliation: Centre for Public Health, National Institute for Health and Care Excellence, London, UK.
authors.forename: Michael P
authors.initials: MP
authors.lastname: Kelly
authors.affiliation: Evidence for Policy and Practice Information and Co-ordinating Centre, Department of Children and Health, Institute of Education, London, UK.
authors.forename: James
authors.initials: J
authors.lastname: Thomas
mesh_terms: D057225:Data Mining / Q000379:methods*; D000069550:Machine Learning; D009323:Natural Language Processing; D010363:Pattern Recognition, Automated / Q000379:methods; D010506:Periodicals as Topic; D018875:Vocabulary, Controlled; D016526:Workload; D000098584:Scoping Reviews as Topic
keywords: scoping review methods; study selection; systematic review methods; text mining