title: Using prediction markets to forecast research evaluations.
abstract: The 2014 Research Excellence Framework (REF2014) was conducted to assess the quality of research carried out at higher education institutions in the UK over a 6 year period. However, the process was criticized for being expensive and bureaucratic, and it was argued that similar information could be obtained more simply from various existing metrics. We were interested in whether a prediction market on the outcome of REF2014 for 33 chemistry departments in the UK would provide information similar to that obtained during the REF2014 process. Prediction markets have become increasingly popular as a means of capturing what is colloquially known as the 'wisdom of crowds', and enable individuals to trade 'bets' on whether a specific outcome will occur or not. These have been shown to be successful at predicting various outcomes in a number of domains (e.g. sport, entertainment and politics), but have rarely been tested against outcomes based on expert judgements such as those that formed the basis of REF2014.
authors.affiliation: MRC Integrative Epidemiology Unit at the University of Bristol , Bristol, UK ; UK Centre for Tobacco and Alcohol Studies, School of Experimental Psychology , University of Bristol , Bristol, UK.
authors.forename: Marcus R
authors.initials: MR
authors.lastname: Munafo
authors.affiliation: New Zealand Institute for Advanced Study , Massey University , Palmerston North, New Zealand ; Wissenschaftskolleg zu Berlin , Berlin, Germany.
authors.forename: Thomas
authors.initials: T
authors.lastname: Pfeiffer
authors.affiliation: Department of Economics , Stockholm School of Economics , Stockholm, Sweden.
authors.forename: Adam
authors.initials: A
authors.lastname: Altmejd
authors.affiliation: Department of Economics , Stockholm School of Economics , Stockholm, Sweden.
authors.forename: Emma
authors.initials: E
authors.lastname: Heikensten
authors.affiliation: Sveriges Riksbank , Stockholm, Sweden.
authors.forename: Johan
authors.initials: J
authors.lastname: Almenberg
authors.affiliation: Department of Philosophy , University of Bristol , Bristol, UK.
authors.forename: Alexander
authors.initials: A
authors.lastname: Bird
authors.affiliation: School of Engineering and Applied Sciences , Harvard University , Cambridge, MA, USA.
authors.forename: Yiling
authors.initials: Y
authors.lastname: Chen
authors.affiliation: Consensus Point , Nashville, TN, USA.
authors.forename: Brad
authors.initials: B
authors.lastname: Wilson
authors.affiliation: Department of Economics , Stockholm School of Economics , Stockholm, Sweden.
authors.forename: Magnus
authors.initials: M
authors.lastname: Johannesson
authors.affiliation: Department of Economics , Stockholm School of Economics , Stockholm, Sweden.
authors.forename: Anna
authors.initials: A
authors.lastname: Dreber
keywords: prediction market; research evaluation; research excellence framework