title: Study on the impact of partition-induced dataset shift on k-fold cross-validation.
abstract: Cross-validation is a very commonly employed technique used to evaluate classifier performance. However, it can potentially introduce dataset shift, a harmful factor that is often not taken into account and can result in inaccurate performance estimation. This paper analyzes the prevalence and impact of partition-induced covariate shift on different k-fold cross-validation schemes. From the experimental results obtained, we conclude that the degree of partition-induced covariate shift depends on the cross-validation scheme considered. In this way, worse schemes may harm the correctness of a single-classifier performance estimation and also increase the needed number of repetitions of cross-validation to reach a stable performance estimation.
authors.forename: Jose García
authors.initials: JG
authors.lastname: Moreno-Torres
authors.forename: José A
authors.initials: JA
authors.lastname: Saez
authors.forename: Francisco
authors.initials: F
authors.lastname: Herrera