title: Mastering the game of Go with deep neural networks and tree search.
abstract: The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses 'value networks' to evaluate board positions and 'policy networks' to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: David
authors.initials: D
authors.lastname: Silver
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Aja
authors.initials: A
authors.lastname: Huang
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Chris J
authors.initials: CJ
authors.lastname: Maddison
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Arthur
authors.initials: A
authors.lastname: Guez
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Laurent
authors.initials: L
authors.lastname: Sifre
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: George
authors.initials: G
authors.lastname: van den Driessche
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Julian
authors.initials: J
authors.lastname: Schrittwieser
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Ioannis
authors.initials: I
authors.lastname: Antonoglou
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Veda
authors.initials: V
authors.lastname: Panneershelvam
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Marc
authors.initials: M
authors.lastname: Lanctot
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Sander
authors.initials: S
authors.lastname: Dieleman
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Dominik
authors.initials: D
authors.lastname: Grewe
authors.affiliation: Google, 1600 Amphitheatre Parkway, Mountain View, California 94043, USA.
authors.forename: John
authors.initials: J
authors.lastname: Nham
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Nal
authors.initials: N
authors.lastname: Kalchbrenner
authors.affiliation: Google, 1600 Amphitheatre Parkway, Mountain View, California 94043, USA.
authors.forename: Ilya
authors.initials: I
authors.lastname: Sutskever
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Timothy
authors.initials: T
authors.lastname: Lillicrap
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Madeleine
authors.initials: M
authors.lastname: Leach
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Koray
authors.initials: K
authors.lastname: Kavukcuoglu
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Thore
authors.initials: T
authors.lastname: Graepel
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Demis
authors.initials: D
authors.lastname: Hassabis
mesh_terms: D003201:Computers; D005060:Europe; D000068957:Games, Recreational; D006801:Humans; D009010:Monte Carlo Method; D016571:Neural Networks, Computer; D012054:Reinforcement, Psychology; D012984:Software; D000069553:Supervised Machine Learning