Online learning and generalization of parts-based image representations by non-negative sparse autoencoders.
We present an efficient online learning scheme for non-negative sparse coding in autoencoder neural networks. It comprises a novel synaptic decay rule that ensures non-negative weights in combination with an intrinsic self-adaptation rule that optimizes sparseness of the non-negative encoding. We show that non-negativity constrains the space of solutions such that overfitting is prevented and very similar encodings are found irrespective of the network initialization and size. We benchmark the novel method on real-world datasets of handwritten digits and faces. The autoencoder yields higher sparseness and lower reconstruction errors than related offline algorithms based on matrix factorization. It generalizes to new inputs both accurately and without costly computations, which is fundamentally different from the classical matrix factorization approaches.
Research Institute for Cognition and Robotics (CoR-Lab), Bielefeld University, Universitätsstr. 25, 33615 Bielefeld, Germany. alemme@CoR-Lab.Uni-Bielefeld.de
Andre
A
Lemme
René Felix
RF
Reinhart
Jochen Jakob
JJ
Steil
D001185:Artificial Intelligence; D016208:Databases, Factual; D005145:Face; D006801:Humans; D007091:Image Processing, Computer-Assisted; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated / Q000379:methods*; D010775:Photic Stimulation / Q000379:methods*