title: Wrapper feature selection for small sample size data driven by complete error estimates.
abstract: This paper focuses on wrapper-based feature selection for a 1-nearest neighbor classifier. We consider in particular the case of a small sample size with a few hundred instances, which is common in biomedical applications. We propose a technique for calculating the complete bootstrap for a 1-nearest-neighbor classifier (i.e., averaging over all desired test/train partitions of the data). The complete bootstrap and the complete cross-validation error estimate with lower variance are applied as novel selection criteria and are compared with the standard bootstrap and cross-validation in combination with three optimization techniques - sequential forward selection (SFS), binary particle swarm optimization (BPSO) and simplified social impact theory based optimization (SSITO). The experimental comparison based on ten datasets draws the following conclusions: for all three search methods examined here, the complete criteria are a significantly better choice than standard 2-fold cross-validation, 10-fold cross-validation and bootstrap with 50 trials irrespective of the selected output number of iterations. All the complete criterion-based 1NN wrappers with SFS search performed better than the widely-used FILTER and SIMBA methods. We also demonstrate the benefits and properties of our approaches on an important and novel real-world application of automatic detection of the subthalamic nucleus.
authors.affiliation: Czech Technical University, Faculty of Electrical Engineering, Department of Cybernetics, Karlovo Namesti 13, 12135 Prague, Czech Republic.
authors.forename: Martin
authors.initials: M
authors.lastname: Macaš
authors.forename: Lenka
authors.initials: L
authors.lastname: Lhotská
authors.forename: Eduard
authors.initials: E
authors.lastname: Bakstein
authors.forename: Daniel
authors.initials: D
authors.lastname: Novák
authors.forename: Jiří
authors.initials: J
authors.lastname: Wild
authors.forename: Tomáš
authors.initials: T
authors.lastname: Sieger
authors.forename: Pavel
authors.initials: P
authors.lastname: Vostatek
authors.forename: Robert
authors.initials: R
authors.lastname: Jech
mesh_terms: D008962:Models, Theoretical; D018401:Sample Size