title: Learning in compressed space.
abstract: We examine two methods which are used to deal with complex machine learning problems: compressed sensing and model compression. We discuss both methods in the context of feed-forward artificial neural networks and develop the backpropagation method in compressed parameter space. We further show that compressing the weights of a layer of a multilayer perceptron is equivalent to compressing the input of the layer. Based on this theoretical framework, we will use orthogonal functions and especially random projections for compression and perform experiments in supervised and reinforcement learning to demonstrate that the presented methods reduce training time significantly.
authors.affiliation: University of Bremen, Fachbereich 3 - Mathematik und Informatik, Postfach 330 440, 28334 Bremen, Germany. afabisch@googlemail.com
authors.forename: Alexander
authors.initials: A
authors.lastname: Fabisch
authors.forename: Yohannes
authors.initials: Y
authors.lastname: Kassahun
authors.forename: Hendrik
authors.initials: H
authors.lastname: Wöhrle
authors.forename: Frank
authors.initials: F
authors.lastname: Kirchner
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D062207:Brain-Computer Interfaces; D003198:Computer Simulation; D044962:Data Compression; D004569:Electroencephalography; D018913:Event-Related Potentials, P300; D006801:Humans; D007858:Learning; D009415:Nerve Net / Q000502:physiology; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D010465:Perception; D011336:Probability