title: Crowdsourcing for cognitive science--the utility of smartphones.
abstract: By 2015, there will be an estimated two billion smartphone users worldwide. This technology presents exciting opportunities for cognitive science as a medium for rapid, large-scale experimentation and data collection. At present, cost and logistics limit most study populations to small samples, restricting the experimental questions that can be addressed. In this study we investigated whether the mass collection of experimental data using smartphone technology is valid, given the variability of data collection outside of a laboratory setting. We presented four classic experimental paradigms as short games, available as a free app and over the first month 20,800 users submitted data. We found that the large sample size vastly outweighed the noise inherent in collecting data outside a controlled laboratory setting, and show that for all four games canonical results were reproduced. For the first time, we provide experimental validation for the use of smartphones for data collection in cognitive science, which can lead to the collection of richer data sets and a significant cost reduction as well as provide an opportunity for efficient phenotypic screening of large populations.
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Harriet R
authors.initials: HR
authors.lastname: Brown
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Peter
authors.initials: P
authors.lastname: Zeidman
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Peter
authors.initials: P
authors.lastname: Smittenaar
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Rick A
authors.initials: RA
authors.lastname: Adams
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom; School of Psychology, University of Birmingham, London, United Kingdom.
authors.forename: Fiona
authors.initials: F
authors.lastname: McNab
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Robb B
authors.initials: RB
authors.lastname: Rutledge
authors.affiliation: Oxford Centre for Human Brain Activity, University of Oxford, Oxford, United Kingdom.
authors.forename: Raymond J
authors.initials: RJ
authors.lastname: Dolan
mesh_terms: D000293:Adolescent; D000328:Adult; D000367:Age Factors; D000368:Aged; D054518:Attentional Blink; D040421:Cell Phone; D019336:Cognitive Science / Q000379:methods*; D063045:Crowdsourcing / Q000379:methods*; D003657:Decision Making; D005260:Female; D006801:Humans; D008297:Male; D008570:Memory, Short-Term; D008875:Middle Aged; D059015:Wireless Technology; D055815:Young Adult