Seriousness checks are useful to improve data validity in online research.
Nonserious answering behavior increases noise and reduces experimental power; it is therefore one of the most important threats to the validity of online research. A simple way to address the problem is to ask respondents about the seriousness of their participation and to exclude self-declared nonserious participants from analysis. To validate this approach, a survey was conducted in the week prior to the German 2009 federal election to the Bundestag. Serious participants answered a number of attitudinal and behavioral questions in a more consistent and predictively valid manner than did nonserious participants. We therefore recommend routinely employing seriousness checks in online surveys to improve data validity.
Department of Experimental Psychology, University of Duesseldorf, Universitaetsstrasse 1, Building 23.03, 40225, Duesseldorf, Germany. frederik.aust@hhu.de
Frederik
F
Aust
Birk
B
Diedenhofen
Sebastian
S
Ullrich
Jochen
J
Musch
D000293:Adolescent; D000328:Adult; D000367:Age Factors; D001290:Attitude; D003627:Data Interpretation, Statistical; D004522:Educational Status; D005260:Female; D005858:Germany; D006801:Humans; D016247:Information Storage and Retrieval / Q000592:standards; D008297:Male; D009862:Online Systems / Q000592:standards*; D018579:Patient Selection; D011057:Politics; D011159:Population Surveillance; D015203:Reproducibility of Results; D012107:Research Design; D057566:Self Report / Q000592:standards*; D011795:Surveys and Questionnaires / Q000592:standards*; D055815:Young Adult