title: Fractional-order gradient descent learning of BP neural networks with Caputo derivative.
abstract: Fractional calculus has been found to be a promising area of research for information processing and modeling of some physical systems. In this paper, we propose a fractional gradient descent method for the backpropagation (BP) training of neural networks. In particular, the Caputo derivative is employed to evaluate the fractional-order gradient of the error defined as the traditional quadratic energy function. The monotonicity and weak (strong) convergence of the proposed approach are proved in detail. Two simulations have been implemented to illustrate the performance of presented fractional-order BP algorithm on three small datasets and one large dataset. The numerical simulations effectively verify the theoretical observations of this paper as well.
authors.affiliation: College of Science, China University of Petroleum, Qingdao, 266580, China. Electronic address: wangjiannl@upc.edu.cn.
authors.forename: Jian
authors.initials: J
authors.lastname: Wang
authors.affiliation: College of Science, China University of Petroleum, Qingdao, 266580, China.
authors.forename: Yanqing
authors.initials: Y
authors.lastname: Wen
authors.affiliation: College of Science, China University of Petroleum, Qingdao, 266580, China.
authors.forename: Yida
authors.initials: Y
authors.lastname: Gou
authors.affiliation: College of Computer & Communication Engineering, China University of Petroleum, Qingdao, 266580, China.
authors.forename: Zhenyun
authors.initials: Z
authors.lastname: Ye
authors.affiliation: College of Science, China University of Petroleum, Qingdao, 266580, China. Electronic address: chenhua@upc.edu.cn.
authors.forename: Hua
authors.initials: H
authors.lastname: Chen
mesh_terms: D000465:Algorithms; D000069550:Machine Learning; D016571:Neural Networks, Computer
keywords: Backpropagation; Caputo derivative; Convergence; Fractional calculus; Monotonicity