title: Temperature based Restricted Boltzmann Machines.
abstract: Restricted Boltzmann machines (RBMs), which apply graphical models to learning probability distribution over a set of inputs, have attracted much attention recently since being proposed as building blocks of multi-layer learning systems called deep belief networks (DBNs). Note that temperature is a key factor of the Boltzmann distribution that RBMs originate from. However, none of existing schemes have considered the impact of temperature in the graphical model of DBNs. In this work, we propose temperature based restricted Boltzmann machines (TRBMs) which reveals that temperature is an essential parameter controlling the selectivity of the firing neurons in the hidden layers. We theoretically prove that the effect of temperature can be adjusted by setting the parameter of the sharpness of the logistic function in the proposed TRBMs. The performance of RBMs can be improved by adjusting the temperature parameter of TRBMs. This work provides a comprehensive insights into the deep belief networks and deep learning architectures from a physical point of view.
authors.affiliation: Center for Brain Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China, 100084.
authors.forename: Guoqi
authors.initials: G
authors.lastname: Li
authors.affiliation: Center for Brain Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China, 100084.
authors.forename: Lei
authors.initials: L
authors.lastname: Deng
authors.affiliation: School of Computing Enginering, Nanyang Technological University, Singapore, 639798.
authors.forename: Yi
authors.initials: Y
authors.lastname: Xu
authors.affiliation: School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798.
authors.forename: Changyun
authors.initials: C
authors.lastname: Wen
authors.affiliation: School of Automation Science and Electrical Engineering, Beihang University, Beijing, China, 100191.
authors.forename: Wei
authors.initials: W
authors.lastname: Wang
authors.affiliation: Center for Brain Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China, 100084.
authors.forename: Jing
authors.initials: J
authors.lastname: Pei
authors.affiliation: Center for Brain Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China, 100084.
authors.forename: Luping
authors.initials: L
authors.lastname: Shi