title: Deep Unsupervised Learning on a Desktop PC: A Primer for Cognitive Scientists.
abstract: Deep belief networks hold great promise for the simulation of human cognition because they show how structured and abstract representations may emerge from probabilistic unsupervised learning. These networks build a hierarchy of progressively more complex distributed representations of the sensory data by fitting a hierarchical generative model. However, learning in deep networks typically requires big datasets and it can involve millions of connection weights, which implies that simulations on standard computers are unfeasible. Developing realistic, medium-to-large-scale learning models of cognition would therefore seem to require expertise in programing parallel-computing hardware, and this might explain why the use of this promising approach is still largely confined to the machine learning community. Here we show how simulations of deep unsupervised learning can be easily performed on a desktop PC by exploiting the processors of low cost graphic cards (graphic processor units) without any specific programing effort, thanks to the use of high-level programming routines (available in MATLAB or Python). We also show that even an entry-level graphic card can outperform a small high-performance computing cluster in terms of learning time and with no loss of learning quality. We therefore conclude that graphic card implementations pave the way for a widespread use of deep learning among cognitive scientists for modeling cognition and behavior.
authors: Computational Cognitive Neuroscience Lab, Department of General Psychology, University of Padova Padova, Italy.
authors: Alberto
authors: A
authors: Testolin
authors: Ivilin
authors: I
authors: Stoianov
authors: Michele
authors: M
authors: De Filippo De Grazia
authors: Marco
authors: M
authors: Zorzi
keywords: GPUs; MPI; cognitive modeling; computer cluster; deep neural networks; hierarchical generative models; parallel-computing architectures; unsupervised learning
references: Neural Comput. 2006 Jul;18(7):1527-54
references: 16764513
references: Cognition. 1995 May;55(2):151-218; discussion 219-26
references: 7789090
references: Neural Netw. 2003 Dec;16(10):1429-51
references: 14622875
references: Neural Comput. 2010 Dec;22(12):3207-20
references: 20858131
references: Wiley Interdiscip Rev Cogn Sci. 2011 Sep;2(5):580-593
references: 26302308
references: Science. 2006 Jul 28;313(5786):504-7
references: 16873662
references: Cogn Process. 2012 Aug;13 Suppl 1:S141-6
references: 22802037
references: Nature. 1996 Jun 13;381(6583):607-9
references: 8637596
references: Behav Brain Sci. 2013 Jun;36(3):181-204
references: 23663408
references: Nat Neurosci. 2012 Jan 08;15(2):194-6
references: 22231428
references: Philos Trans R Soc Lond B Biol Sci. 1997 Aug 29;352(1358):1177-90
references: 9304685
references: Trends Cogn Sci. 1998 Nov 1;2(11):455-62
references: 21227277
references: Nat Neurosci. 1999 Jan;2(1):79-87
references: 10195184
references: Philos Trans R Soc Lond B Biol Sci. 2010 Jan 12;365(1537):177-84
references: 20008395
references: Trends Cogn Sci. 2007 Oct;11(10):428-34
references: 17921042
references: Neural Comput. 2002 Aug;14(8):1771-800
references: 12180402
references: Cogn Psychol. 2010 Sep;61(2):106-51
references: 20510406
references: Neural Netw. 2008 Oct;21(8):1146-52
references: 18684591