title: Conducting Online Behavioral Research Using Crowdsourcing Services in Japan.
abstract: Recent research on human behavior has often collected empirical data from the online labor market, through a process known as crowdsourcing. As well as the United States and the major European countries, there are several crowdsourcing services in Japan. For research purpose, Amazon's Mechanical Turk (MTurk) is the widely used platform among those services. Previous validation studies have shown many commonalities between MTurk workers and participants from traditional samples based on not only personality but also performance on reasoning tasks. The present study aims to extend these findings to non-MTurk (i.e., Japanese) crowdsourcing samples in which workers have different ethnic backgrounds from those of MTurk. We conducted three surveys (N= 426, 453, 167, respectively) designed to compare Japanese crowdsourcing workers and university students in terms of their demographics, personality traits, reasoning skills, and attention to instructions. The results generally align with previous studies and suggest that non-MTurk participants are also eligible for behavioral research. Furthermore, small screen devices are found to impair participants' attention to instructions. Several recommendations concerning this sample are presented.
authors.affiliation: Department of Psychology for Well-Being, Hokusei Gakuen University Sapporo, Japan.
authors.forename: Yoshimasa
authors.initials: Y
authors.lastname: Majima
authors.affiliation: Department of Psychology for Well-Being, Hokusei Gakuen University Sapporo, Japan.
authors.forename: Kaoru
authors.initials: K
authors.lastname: Nishiyama
authors.affiliation: Department of Foreign Language Education, Hokusei Gakuen University Sapporo, Japan.
authors.forename: Aki
authors.initials: A
authors.lastname: Nishihara
authors.affiliation: Department of Social Work, Hokusei Gakuen University Sapporo, Japan.
authors.forename: Ryosuke
authors.initials: R
authors.lastname: Hata
keywords: instructional manipulation check; non-MTurk crowdsourcing; online study; personality; reasoning