A universal learning rule that minimizes well-formed cost functions.
In this paper, we analyze stochastic gradient learning rules for posterior probability estimation using networks with a single layer of weights and a general nonlinear activation function. We provide necessary and sufficient conditions on the learning rules and the activation function to obtain probability estimates. Also, we extend the concept of well-formed cost function, proposed by Wittner and Denker, to multiclass problems, and we provide theoretical results showing the advantages of this kind of objective functions.
Mora-Jiménez
Inma
I
Department of Signal Theory and Communications, University Carlos III de Madrid, 28911 Leganés-Madrid, Spain. inmoji@tsc.uc3m.es
Cid-Sueiro
Jesús
J
D000465:Algorithms; D016000:Cluster Analysis; D003198:Computer Simulation; D003205:Computing Methodologies; D003661:Decision Support Techniques; D008954:Models, Biological; D015233:Models, Statistical; D016571:Neural Networks, Computer; D009716:Numerical Analysis, Computer-Assisted; D010363:Pattern Recognition, Automated / Q000379:methods*; D013269:Stochastic Processes