title: Hybrid computing using a neural network with dynamic external memory.
abstract: Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read-write memory.
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Alex
authors.initials: A
authors.lastname: Graves
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Greg
authors.initials: G
authors.lastname: Wayne
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Malcolm
authors.initials: M
authors.lastname: Reynolds
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Tim
authors.initials: T
authors.lastname: Harley
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Ivo
authors.initials: I
authors.lastname: Danihelka
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Agnieszka
authors.initials: A
authors.lastname: Grabska-Barwińska
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Sergio Gómez
authors.initials: SG
authors.lastname: Colmenarejo
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Edward
authors.initials: E
authors.lastname: Grefenstette
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Tiago
authors.initials: T
authors.lastname: Ramalho
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: John
authors.initials: J
authors.lastname: Agapiou
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Adrià Puigdomènech
authors.initials: AP
authors.lastname: Badia
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Karl Moritz
authors.initials: KM
authors.lastname: Hermann
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Yori
authors.initials: Y
authors.lastname: Zwols
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Georg
authors.initials: G
authors.lastname: Ostrovski
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Adam
authors.initials: A
authors.lastname: Cain
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Helen
authors.initials: H
authors.lastname: King
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Christopher
authors.initials: C
authors.lastname: Summerfield
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Phil
authors.initials: P
authors.lastname: Blunsom
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Koray
authors.initials: K
authors.lastname: Kavukcuoglu
authors.affiliation: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.
authors.forename: Demis
authors.initials: D
authors.lastname: Hassabis