title: Review: Are we stumbling in our quest to find the best predictor? Over-optimism in sensor-based models for predicting falls in older adults.
abstract: The field of fall risk testing using wearable sensors is bustling with activity. In this Letter, the authors review publications which incorporated features extracted from sensor signals into statistical models intended to estimate fall risk or predict falls in older people. A review of these studies raises concerns that this body of literature is presenting over-optimistic results in light of small sample sizes, questionable modelling decisions and problematic validation methodologies (e.g. inherent problems with the overly-popular cross-validation technique, lack of external validation). There seem to be substantial issues in the feature selection process, whereby researchers select features before modelling begins based on their relation to the target, and either perform no validation or test the models on the same data used for their training. This, together with potential issues related to the large number of features and their correlations, inevitably leads to models with inflated accuracy that are unlikely to maintain their reported performance during everyday use in relevant populations. Indeed, the availability of rich sensor data and many analytical options provides intellectual and creative freedom for researchers, but should be treated with caution, and such pitfalls must be avoided if we desire to create generalisable prognostic tools of any clinical value.
authors.affiliation: Graduate School of Biomedical Engineering , University of New South Wales , Sydney 2052 , Australia.
authors.forename: Tal
authors.initials: T
authors.lastname: Shany
authors.affiliation: Graduate School of Biomedical Engineering , University of New South Wales , Sydney 2052 , Australia.
authors.forename: Kejia
authors.initials: K
authors.lastname: Wang
authors.affiliation: Graduate School of Biomedical Engineering , University of New South Wales , Sydney 2052 , Australia.
authors.forename: Ying
authors.initials: Y
authors.lastname: Liu
authors.affiliation: Graduate School of Biomedical Engineering , University of New South Wales , Sydney 2052 , Australia.
authors.forename: Nigel H
authors.initials: NH
authors.lastname: Lovell
authors.affiliation: Graduate School of Biomedical Engineering , University of New South Wales , Sydney 2052 , Australia.
authors.forename: Stephen J
authors.initials: SJ
authors.lastname: Redmond
keywords: biomechanics; body sensor networks; cross-validation technique; fall prediction; fall risk testing; feature extraction; feature selection; geriatrics; medical signal processing; older adults; patient monitoring; prognostic tools; review; reviews; sensor-based models; statistical models; telemedicine; wearable sensors