title: A universal learning rule that minimizes well-formed cost functions.
abstract: In this paper, we analyze stochastic gradient learning rules for posterior probability estimation using networks with a single layer of weights and a general nonlinear activation function. We provide necessary and sufficient conditions on the learning rules and the activation function to obtain probability estimates. Also, we extend the concept of well-formed cost function, proposed by Wittner and Denker, to multiclass problems, and we provide theoretical results showing the advantages of this kind of objective functions.
authors.affiliation: Department of Signal Theory and Communications, University Carlos III de Madrid, 28911 Leganés-Madrid, Spain. inmoji@tsc.uc3m.es
authors.forename: Inma
authors.initials: I
authors.lastname: Mora-Jiménez
authors.forename: Jesús
authors.initials: J
authors.lastname: Cid-Sueiro
mesh_terms: D000465:Algorithms; D016000:Cluster Analysis; D003198:Computer Simulation; D003205:Computing Methodologies; D003661:Decision Support Techniques; D008954:Models, Biological; D015233:Models, Statistical; D016571:Neural Networks, Computer; D009716:Numerical Analysis, Computer-Assisted; D010363:Pattern Recognition, Automated / Q000379:methods*; D013269:Stochastic Processes