title: Accuracy and consistency of grass pollen identification by human analysts using electron micrographs of surface ornamentation.
abstract: OBJECTIVE
  Humans frequently identify pollen grains at a taxonomic rank above species. Grass pollen is a classic case of this situation, which has led to the development of computational methods for identifying grass pollen species. This paper aims to provide context for these computational methods by quantifying the accuracy and consistency of human identification. •
  
  
  METHODS
  We measured the ability of nine human analysts to identify 12 species of grass pollen using scanning electron microscopy images. These are the same images that were used in computational identifications. We have measured the coverage, accuracy, and consistency of each analyst, and investigated their ability to recognize duplicate images. •
  
  
  RESULTS
  Coverage ranged from 87.5% to 100%. Mean identification accuracy ranged from 46.67% to 87.5%. The identification consistency of each analyst ranged from 32.5% to 87.5%, and each of the nine analysts produced considerably different identification schemes. The proportion of duplicate image pairs that were missed ranged from 6.25% to 58.33%. •
  
  
  CONCLUSIONS
  The identification errors made by each analyst, which result in a decline in accuracy and consistency, are likely related to psychological factors such as the limited capacity of human memory, fatigue and boredom, recency effects, and positivity bias.
authors.affiliation: College of Life and Environmental Sciences, University of Exeter, Prince of Wales Road, Exeter, Devon EX4 4PS, United Kingdom.
authors.forename: Luke
authors.initials: L
authors.lastname: Mander
authors.affiliation: College of Life and Environmental Sciences, University of Exeter, Prince of Wales Road, Exeter, Devon EX4 4PS, United Kingdom.
authors.forename: Sarah J
authors.initials: SJ
authors.lastname: Baker
authors.affiliation: College of Life and Environmental Sciences, University of Exeter, Prince of Wales Road, Exeter, Devon EX4 4PS, United Kingdom.
authors.forename: Claire M
authors.initials: CM
authors.lastname: Belcher
authors.affiliation: Program in Ecology, Evolution, and Conservation Biology, University of Illinois, 505 South Goodwin Avenue, Urbana, Illinois 61801 USA.
authors.forename: Derek S
authors.initials: DS
authors.lastname: Haselhorst
authors.affiliation: Department of Plant Biology, University of Illinois, 505 South Goodwin Avenue, Urbana, Illinois 61801 USA.
authors.forename: Jacklyn
authors.initials: J
authors.lastname: Rodriguez
authors.affiliation: College of Life and Environmental Sciences, University of Exeter, Prince of Wales Road, Exeter, Devon EX4 4PS, United Kingdom.
authors.forename: Jessica L
authors.initials: JL
authors.lastname: Thorn
authors.affiliation: Department of Earth Sciences, Indian Institute of Technology Roorkee, Roorkee 247667, Uttarakhand, India.
authors.forename: Shivangi
authors.initials: S
authors.lastname: Tiwari
authors.affiliation: College of Life and Environmental Sciences, University of Exeter, Prince of Wales Road, Exeter, Devon EX4 4PS, United Kingdom.
authors.forename: Dunia H
authors.initials: DH
authors.lastname: Urrego
authors.affiliation: Program in Ecology, Evolution, and Conservation Biology, University of Illinois, 505 South Goodwin Avenue, Urbana, Illinois 61801 USA.
authors.forename: Cassandra J
authors.initials: CJ
authors.lastname: Wesseln
authors.affiliation: Department of Plant Biology, University of Illinois, 505 South Goodwin Avenue, Urbana, Illinois 61801 USA.
authors.forename: Surangi W
authors.initials: SW
authors.lastname: Punyasena
keywords: automation; classification; expert analysis; identification; palynology