title: A benchmark test of accuracy and precision in estimating dynamical systems characteristics from a time series.
abstract: Characteristics of dynamical systems are often estimated to describe physiological processes. For instance, Lyapunov exponents have been determined to assess the stability of the cardio-vascular system, respiration, and, more recently, human gait and posture. However, the systematic evaluation of the accuracy and precision of these estimates is problematic because the proper values of the characteristics are typically unknown. We fill this void with a set of standardized time series with well-defined dynamical characteristics that serve as a benchmark. Estimates ought to match these characteristics, at least to good approximation. We outline a procedure to employ this generic benchmark test and illustrate its capacity by examining methods for estimating the maximum Lyapunov exponent. In particular, we discuss algorithms by Wolf and co-workers and by Rosenstein and co-workers and evaluate their performances as a function of signal length and signal-to-noise ratio. In all scenarios, the precision of Rosenstein's algorithm was found to be equal to or greater than Wolf's algorithm. The latter, however, appeared more accurate if reasonably large signal lengths are available and noise levels are sufficiently low. Due to its modularity, the presented benchmark test can be used to evaluate and tune any estimation method to perform optimally for arbitrary experimental data.
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands.
authors.forename: S M
authors.initials: SM
authors.lastname: Rispens
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands.
authors.forename: M
authors.initials: M
authors.lastname: Pijnappels
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands; King Abdulaziz University, Jeddah, Saudi Arabia.
authors.forename: J H
authors.initials: JH
authors.lastname: van Dieën
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands.
authors.forename: K S
authors.initials: KS
authors.lastname: van Schooten
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands; Brunel University, School of Sport & Education, Uxbridge, UK.
authors.forename: P J
authors.initials: PJ
authors.lastname: Beek
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Human Movement Sciences, VU University Amsterdam, van der Boechorststraat 9, 1081 BT Amsterdam, The Netherlands. Electronic address: a.daffertshofer@vu.nl.
authors.forename: A
authors.initials: A
authors.lastname: Daffertshofer
mesh_terms: D000465:Algorithms; D019985:Benchmarking; D005684:Gait / Q000502:physiology*; D006801:Humans; D011187:Posture / Q000502:physiology*
keywords: Lyapunov exponent; Physiological systems; Sample size; Signal-to-noise ratio; Time series