title: Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves?
abstract: Volunteers are increasingly being recruited into citizen science projects to collect observations for scientific studies. An additional goal of these projects is to engage and educate these volunteers. Thus, there are few barriers to participation resulting in volunteer observers with varying ability to complete the project's tasks. To improve the quality of a citizen science project's outcomes it would be useful to account for inter-observer variation, and to assess the rarely tested presumption that participating in a citizen science projects results in volunteers becoming better observers. Here we present a method for indexing observer variability based on the data routinely submitted by observers participating in the citizen science project eBird, a broad-scale monitoring project in which observers collect and submit lists of the bird species observed while birding. Our method for indexing observer variability uses species accumulation curves, lines that describe how the total number of species reported increase with increasing time spent in collecting observations. We find that differences in species accumulation curves among observers equates to higher rates of species accumulation, particularly for harder-to-identify species, and reveals increased species accumulation rates with continued participation. We suggest that these properties of our analysis provide a measure of observer skill, and that the potential to derive post-hoc data-derived measurements of participant ability should be more widely explored by analysts of data from citizen science projects. We see the potential for inferential results from analyses of citizen science data to be improved by accounting for observer skill.
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Steve
authors.initials: S
authors.lastname: Kelling
authors.affiliation: British Trust for Ornithology, Thetford, Norfolk, England, United Kingdom.
authors.forename: Alison
authors.initials: A
authors.lastname: Johnston
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Wesley M
authors.initials: WM
authors.lastname: Hochachka
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Marshall
authors.initials: M
authors.lastname: Iliff
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Daniel
authors.initials: D
authors.lastname: Fink
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Jeff
authors.initials: J
authors.lastname: Gerbracht
authors.affiliation: School of Information, University of Michigan, Ann Arbor, Michigan, United States of America.
authors.forename: Carl
authors.initials: C
authors.lastname: Lagoze
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Frank A
authors.initials: FA
authors.lastname: La Sorte
authors.affiliation: School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, United States of America.
authors.forename: Travis
authors.initials: T
authors.lastname: Moore
authors.affiliation: College of Information Studies, University of Maryland, College Park, Maryland, United States of America.
authors.forename: Andrea
authors.initials: A
authors.lastname: Wiggins
authors.affiliation: School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, United States of America.
authors.forename: Weng-Keen
authors.initials: WK
authors.lastname: Wong
authors.affiliation: Cornell Lab of Ornithology, Cornell University, Ithaca, New York, United States of America.
authors.forename: Chris
authors.initials: C
authors.lastname: Wood
authors.affiliation: School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, United States of America.
authors.forename: Jun
authors.initials: J
authors.lastname: Yu
mesh_terms: D063147:Animal Distribution; D000818:Animals; D001717:Birds / Q000145:classification / Q000502:physiology; D003625:Data Collection / Q000379:methods; D006801:Humans; D015588:Observer Variation; D012586:Science / Q000379:methods*; D014838:Volunteers