title: A solution to the single-question crowd wisdom problem.
abstract: Once considered provocative, the notion that the wisdom of the crowd is superior to any individual has become itself a piece of crowd wisdom, leading to speculation that online voting may soon put credentialed experts out of business. Recent applications include political and economic forecasting, evaluating nuclear safety, public policy, the quality of chemical probes, and possible responses to a restless volcano. Algorithms for extracting wisdom from the crowd are typically based on a democratic voting procedure. They are simple to apply and preserve the independence of personal judgment. However, democratic methods have serious limitations. They are biased for shallow, lowest common denominator information, at the expense of novel or specialized knowledge that is not widely shared. Adjustments based on measuring confidence do not solve this problem reliably. Here we propose the following alternative to a democratic vote: select the answer that is more popular than people predict. We show that this principle yields the best answer under reasonable assumptions about voter behaviour, while the standard 'most popular' or 'most confident' principles fail under exactly those same assumptions. Like traditional voting, the principle accepts unique problems, such as panel decisions about scientific or artistic merit, and legal or historical disputes. The potential application domain is thus broader than that covered by machine learning and psychometric methods, which require data across multiple questions.
authors.affiliation: Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA.|Department of Economics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA.|Department of Brain &Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA.
authors.forename: Dražen
authors.initials: D
authors.lastname: Prelec
authors.affiliation: Princeton Neuroscience Institute and Computer Science Department, Princeton University, Princeton, New Jersey 08544, USA.
authors.forename: H Sebastian
authors.initials: HS
authors.lastname: Seung
authors.affiliation: Department of Brain &Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA.
authors.forename: John
authors.initials: J
authors.lastname: McCoy
mesh_terms: D000465:Algorithms; D001154:Art; D001499:Bayes Theorem; D002755:Choice Behavior; D003132:Commerce; D003709:Democracy; D005104:Expert Testimony; D005843:Geography; D006801:Humans; D007600:Judgment; D019359:Knowledge; D016015:Logistic Models; D011639:Public Opinion