title: The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange.
abstract: Crowds can often make better decisions than individuals or small groups of experts by leveraging their ability to aggregate diverse information. Question answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect to identify the best answers to questions asked by users. We analyze data from 250 communities on the Stack Exchange network to pinpoint factors affecting which answers are chosen as the best answers. Our results suggest that, rather than evaluate all available answers to a question, users rely on simple cognitive heuristics to choose an answer to vote for or accept. These cognitive heuristics are linked to an answer's salience, such as the order in which it is listed and how much screen space it occupies. While askers appear to depend on heuristics to a greater extent than voters when choosing an answer to accept as the most helpful one, voters use acceptance itself as a heuristic, and they are more likely to choose the answer after it has been accepted than before that answer was accepted. These heuristics become more important in explaining and predicting behavior as the number of available answers to a question increases. Our findings suggest that crowd judgments may become less reliable as the number of answers grows.
authors.affiliation: Dept of Computer Science, University of California at Davis, Davis, CA, United States of America.|Dept of Political Science, University of California at Davis, Davis, CA, United States of America.
authors.forename: Keith
authors.initials: K
authors.lastname: Burghardt
authors.affiliation: University of Modena and Reggio Emilia, Modena MO, Italy.
authors.forename: Emanuel F
authors.initials: EF
authors.lastname: Alsina
authors.affiliation: Dept of Physics, University of Maryland, College Park, MD, United States of America.|Santa Fe Institute, Santa Fe, NM, United States of America.
authors.forename: Michelle
authors.initials: M
authors.lastname: Girvan
authors.affiliation: Department of Business Management, North Carolina State University, Raleigh, NC, United States of America.
authors.forename: William
authors.initials: W
authors.lastname: Rand
authors.affiliation: Information Sciences Institute, University of Southern California, Marina del Rey, CA, United States of America.
authors.forename: Kristina
authors.initials: K
authors.lastname: Lerman
mesh_terms: D001519:Behavior; D003071:Cognition; D003441:Crowding; D006801:Humans; D016015:Logistic Models