title: Approximate Bayesian MLP regularization for regression in the presence of noise.
abstract: We present a novel regularization method for a multilayer perceptron (MLP) that learns a regression function in the presence of noise regardless of how smooth the function is. Unlike general MLP regularization methods assuming that a regression function is smooth, the proposed regularization method is also valid when a regression function has discontinuities (non-smoothness). Since a true regression function to be learned is unknown, we examine a training set with our Bayesian approach that identifies non-smooth data, analyzing discontinuities in a regression function. The use of a Bayesian probability distribution identifies the non-smooth data. These identified data is used in a proposed objective function to fit an MLP response to the desired regression function regardless of its smoothness and noise. Experimental simulations show that the MLP with our presented training method yields more accurate fits to non-smooth functions than other MLP training methods. Further, we show that the suggested training methodology can be incorporated with deep learning models.
authors: School of Computing, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 305-701, Republic of Korea. Electronic address: jgparknn@kaist.ac.kr.
authors: Jung-Guk
authors: JG
authors: Park
authors: School of Computing, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 305-701, Republic of Korea. Electronic address: shjo@kaist.ac.kr.
authors: Sungho
authors: S
authors: Jo
mesh_terms: D001499:Bayes Theorem; D000069550:Machine Learning; D008962:Models, Theoretical; D016571:Neural Networks, Computer
keywords: Bayesian method; Multilayer perceptron training; Non-smooth regression; Regularization; Weight-decay