title: Solving Multiple Isolated, Interleaved, and Blended Tasks through Modular Neuroevolution.
abstract: Many challenging sequential decision-making problems require agents to master multiple tasks. For instance, game agents may need to gather resources, attack opponents, and defend against attacks. Learning algorithms can thus benefit from having separate policies for these tasks, and from knowing when each one is appropriate. How well this approach works depends on how tightly coupled the tasks are. Three cases are identified: Isolated tasks have distinct semantics and do not interact, interleaved tasks have distinct semantics but do interact, and blended tasks have regions where semantics from multiple tasks overlap. Learning across multiple tasks is studied in this article with Modular Multiobjective NEAT, a neuroevolution framework applied to three variants of the challenging Ms. Pac-Man video game. In the standard blended version of the game, a surprising, highly effective machine-discovered task division surpasses human-specified divisions, achieving the best scores to date in this game. In isolated and interleaved versions of the game, human-specified task divisions are also successful, though the best scores are surprisingly still achieved by machine discovery. Modular neuroevolution is thus shown to be capable of finding useful, unexpected task divisions better than those apparent to a human designer.
authors.affiliation: Department of Mathematics and Computer Science, Southwestern University, Georgetown, TX 78626, USA schrum2@southwestern.edu.
authors.forename: Jacob
authors.initials: J
authors.lastname: Schrum
authors.affiliation: Department of Computer Science, The University of Texas at Austin, Austin, TX 78712, USA risto@cs.utexas.edu.
authors.forename: Risto
authors.initials: R
authors.lastname: Miikkulainen
mesh_terms: D001185:Artificial Intelligence; D005075:Biological Evolution; D006801:Humans; D007858:Learning; D008390:Markov Chains; D011340:Problem Solving; D013647:Task Performance and Analysis; D018910:Video Games
keywords: Ms. Pac-Man; Neuroevolution; modularity; multimodal behavior; video games