title: Sparsity Is Better with Stability: Combining Accuracy and Stability for Model Selection in Brain Decoding.
abstract: Structured sparse methods have received significant attention in neuroimaging. These methods allow the incorporation of domain knowledge through additional spatial and temporal constraints in the predictive model and carry the promise of being more interpretable than non-structured sparse methods, such as LASSO or Elastic Net methods. However, although sparsity has often been advocated as leading to more interpretable models it can also lead to unstable models under subsampling or slight changes of the experimental conditions. In the present work we investigate the impact of using stability/reproducibility as an additional model selection criterion on several different sparse (and structured sparse) methods that have been recently applied for fMRI brain decoding. We compare three different model selection criteria: (i) classification accuracy alone; (ii) classification accuracy and overlap between the solutions; (iii) classification accuracy and correlation between the solutions. The methods we consider include LASSO, Elastic Net, Total Variation, sparse Total Variation, Laplacian and Graph Laplacian Elastic Net (GraphNET). Our results show that explicitly accounting for stability/reproducibility during the model optimization can mitigate some of the instability inherent in sparse methods. In particular, using accuracy and overlap between the solutions as a joint optimization criterion can lead to solutions that are more similar in terms of accuracy, sparsity levels and coefficient maps even when different sparsity methods are considered.
authors.affiliation: Laboratory for Information and Inference Systems, École Polytechnique Fédérale de Lausanne (EPFL) Lausanne, Switzerland.
authors.forename: Luca
authors.initials: L
authors.lastname: Baldassarre
authors.affiliation: Istituto Italiano di TecnologiaGenoa, Italy; Department of Computer Science, University College LondonLondon, UK.
authors.forename: Massimiliano
authors.initials: M
authors.lastname: Pontil
authors.affiliation: Department of Computer Science, University College LondonLondon, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College LondonLondon, UK.
authors.forename: Janaina
authors.initials: J
authors.lastname: Mourão-Miranda
keywords: model selection; predictive models; reproducibility; sparse methods; structured sparsity