title: Adaptive initial step size selection for Simultaneous Perturbation Stochastic Approximation.
abstract: A difficulty in using Simultaneous Perturbation Stochastics Approximation (SPSA) is its performance sensitivity to the step sizes chosen at the initial stage of the iteration. If the step size is too large, the solution estimate may fail to converge. The proposed adaptive stepping method automatically reduces the initial step size of the SPSA so that reduction of the objective function value occurs more reliably. Ten mathematical functions each with three different noise levels were used to empirically show the effectiveness of the proposed idea. A parameter estimation example of a nonlinear dynamical system is also included.
authors.affiliation: Ghent University - iMinds, INTEC, Gaston Crommenlaan 8 bus 201, Ledeberg, 9050 Ghent, Belgium ; Noesis Solutions, Gaston Geenslaan 11 B4, 3001 Louvain, Belgium.
authors.forename: Keiichi
authors.identifier: 0000-0003-1364-9477
authors.initials: K
authors.lastname: Ito
authors.affiliation: Ghent University - iMinds, INTEC, Gaston Crommenlaan 8 bus 201, Ledeberg, 9050 Ghent, Belgium.
authors.forename: Tom
authors.initials: T
authors.lastname: Dhaene
keywords: Direct method; Noisy function; Optimization; Parameter estimation; Stochastic approximation