title: psiTurk: An open-source framework for conducting replicable behavioral experiments online.
abstract: Online data collection has begun to revolutionize the behavioral sciences. However, conducting carefully controlled behavioral experiments online introduces a number of new of technical and scientific challenges. The project described in this paper, psiTurk, is an open-source platform which helps researchers develop experiment designs which can be conducted over the Internet. The tool primarily interfaces with Amazon's Mechanical Turk, a popular crowd-sourcing labor market. This paper describes the basic architecture of the system and introduces new users to the overall goals. psiTurk aims to reduce the technical hurdles for researchers developing online experiments while improving the transparency and collaborative nature of the behavioral sciences.
authors.affiliation: Department of Psychology, New York University, New York, NY, USA. todd.gureckis@nyu.edu.
authors.forename: Todd M
authors.initials: TM
authors.lastname: Gureckis
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.|Stitch Fix, Inc., San Francisco, CA, USA.
authors.forename: Jay
authors.initials: J
authors.lastname: Martin
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.|Square, Inc., San Francisco, CA, USA.
authors.forename: John
authors.initials: J
authors.lastname: McDonnell
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.
authors.forename: Alexander S
authors.initials: AS
authors.lastname: Rich
authors.affiliation: Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany.
authors.forename: Doug
authors.initials: D
authors.lastname: Markant
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.
authors.forename: Anna
authors.initials: A
authors.lastname: Coenen
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.
authors.forename: David
authors.initials: D
authors.lastname: Halpern
authors.affiliation: Department of Psychology, University of California at Berkeley, Berkeley, CA, USA.
authors.forename: Jessica B
authors.initials: JB
authors.lastname: Hamrick
authors.affiliation: Department of Psychology, New York University, New York, NY, USA.
authors.forename: Patricia
authors.initials: P
authors.lastname: Chan
mesh_terms: D035841:Behavioral Research / Q000379:methods*; D063045:Crowdsourcing; D003625:Data Collection / Q000379:methods*; D020407:Internet; D012107:Research Design
keywords: Amazon Mechanical Turk; Internet experiments; Online experiments; Open science; psiTurk