title: Identifying and cultivating superforecasters as a method of improving probabilistic predictions.
abstract: Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters. Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created-and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.
authors.affiliation: Department of Psychology, University of Pennsylvania mellers@wharton.upenn.edu.
authors.forename: Barbara
authors.initials: B
authors.lastname: Mellers
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Eric
authors.initials: E
authors.lastname: Stone
authors.affiliation: Haas School of Business, University of California, Berkeley.
authors.forename: Terry
authors.initials: T
authors.lastname: Murray
authors.affiliation: Department of Statistics, Temple University.
authors.forename: Angela
authors.initials: A
authors.lastname: Minster
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Nick
authors.initials: N
authors.lastname: Rohrbaugh
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Michael
authors.initials: M
authors.lastname: Bishop
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Eva
authors.initials: E
authors.lastname: Chen
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Joshua
authors.initials: J
authors.lastname: Baker
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Yuan
authors.initials: Y
authors.lastname: Hou
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Michael
authors.initials: M
authors.lastname: Horowitz
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Lyle
authors.initials: L
authors.lastname: Ungar
authors.affiliation: Department of Psychology, University of Pennsylvania.
authors.forename: Philip
authors.initials: P
authors.lastname: Tetlock
mesh_terms: D019540:Area Under Curve; D003071:Cognition; D004777:Environment; D005544:Forecasting / Q000379:methods; D006801:Humans; D007600:Judgment; D007858:Learning; D008960:Models, Psychological; D009042:Motivation; D011336:Probability; D012372:ROC Curve; D013995:Time
keywords: expertise; forecasts; predictions; probability training; teams