title: Multiagent cooperation and competition with deep reinforcement learning.
abstract: Evolution of cooperation and competition can appear when multiple adaptive agents share a biological, social, or technological niche. In the present work we study how cooperation and competition emerge between autonomous agents that learn by reinforcement while using only their raw visual input as the state representation. In particular, we extend the Deep Q-Learning framework to multiagent environments to investigate the interaction between two learning agents in the well-known video game Pong. By manipulating the classical rewarding scheme of Pong we show how competitive and collaborative behaviors emerge. We also describe the progression from competitive to collaborative behavior when the incentive to cooperate is increased. Finally we show how learning by playing against another adaptive agent, instead of against a hard-wired algorithm, results in more robust strategies. The present work shows that Deep Q-Networks can become a useful tool for studying decentralized learning of multiagent systems coping with high-dimensional environments.
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Ardi
authors.initials: A
authors.lastname: Tampuu
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Tambet
authors.initials: T
authors.lastname: Matiisen
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Dorian
authors.initials: D
authors.lastname: Kodelja
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Ilya
authors.initials: I
authors.lastname: Kuzovkin
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Kristjan
authors.initials: K
authors.lastname: Korjus
authors.affiliation: Department of Mathematics, ETH Zürich, Zürich, Switzerland.
authors.forename: Juhan
authors.initials: J
authors.lastname: Aru
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Jaan
authors.initials: J
authors.lastname: Aru
authors.affiliation: Computational Neuroscience Lab, Institute of Computer Science, University of Tartu, Tartu, Estonia.
authors.forename: Raul
authors.initials: R
authors.lastname: Vicente
mesh_terms: D000465:Algorithms; D003299:Cooperative Behavior; D005716:Game Theory; D006801:Humans; D007398:Interpersonal Relations; D007858:Learning / Q000502:physiology*; D012054:Reinforcement, Psychology; D012201:Reward; D012919:Social Behavior