title: QRTEngine: An easy solution for running online reaction time experiments using Qualtrics.
abstract: Performing online behavioral research is gaining increased popularity among researchers in psychological and cognitive science. However, the currently available methods for conducting online reaction time experiments are often complicated and typically require advanced technical skills. In this article, we introduce the Qualtrics Reaction Time Engine (QRTEngine), an open-source JavaScript engine that can be embedded in the online survey development environment Qualtrics. The QRTEngine can be used to easily develop browser-based online reaction time experiments with accurate timing within current browser capabilities, and it requires only minimal programming skills. After introducing the QRTEngine, we briefly discuss how to create and distribute a Stroop task. Next, we describe a study in which we investigated the timing accuracy of the engine under different processor loads using external chronometry. Finally, we show that the QRTEngine can be used to reproduce classic behavioral effects in three reaction time paradigms: a Stroop task, an attentional blink task, and a masked-priming task. These findings demonstrate that QRTEngine can be used as a tool for conducting online behavioral research even when this requires accurate stimulus presentation times.
authors.affiliation: Institute of Psychology, Leiden University, Wassenaarseweg 52, Leiden, 2333, AK, The Netherlands.|Department of Cognitive Psychology and Ergonomics, Faculty of Behavioral, Management and Social Sciences, University of Twente, Enschede, The Netherlands.|MIRA Research Institute, University of Twente, Enschede, The Netherlands.
authors.forename: Jonathan S
authors.initials: JS
authors.lastname: Barnhoorn
authors.affiliation: Institute of Psychology, Leiden University, Wassenaarseweg 52, Leiden, 2333, AK, The Netherlands.|University of Sheffield, Sheffield, UK.
authors.forename: Erwin
authors.initials: E
authors.lastname: Haasnoot
authors.affiliation: Institute of Psychology, Leiden University, Wassenaarseweg 52, Leiden, 2333, AK, The Netherlands.|Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands.
authors.forename: Bruno R
authors.initials: BR
authors.lastname: Bocanegra
authors.affiliation: Institute of Psychology, Leiden University, Wassenaarseweg 52, Leiden, 2333, AK, The Netherlands. HvanSteenbergen@fsw.leidenuniv.nl.|Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands. HvanSteenbergen@fsw.leidenuniv.nl.
authors.forename: Henk
authors.initials: H
authors.lastname: van Steenbergen
mesh_terms: D035841:Behavioral Research / Q000379:methods*; D006801:Humans; D020407:Internet; D011930:Reaction Time / Q000502:physiology*
keywords: Amazon Mechanical Turk; JavaScript; Online experiments; Open-source; Qualtrics