title: Simple and Effective Way for Data Preprocessing Selection Based on Design of Experiments.
abstract: The selection of optimal preprocessing is among the main bottlenecks in chemometric data analysis. Preprocessing currently is a burden, since a multitude of different preprocessing methods is available for, e.g., baseline correction, smoothing, and alignment, but it is not clear beforehand which method(s) should be used for which data set. The process of preprocessing selection is often limited to trial-and-error and is therefore considered somewhat subjective. In this paper, we present a novel, simple, and effective approach for preprocessing selection. The defining feature of this approach is a design of experiments. On the basis of the design, model performance of a few well-chosen preprocessing methods, and combinations thereof (called strategies) is evaluated. Interpretation of the main effects and interactions subsequently enables the selection of an optimal preprocessing strategy. The presented approach is applied to eight different spectroscopic data sets, covering both calibration and classification challenges. We show that the approach is able to select a preprocessing strategy which improves model performance by at least 50% compared to the raw data; in most cases, it leads to a strategy very close to the true optimum. Our approach makes preprocessing selection fast, insightful, and objective.
authors.affiliation: Radboud University , Institute for Molecules and Materials, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands.|TI-COAST , Science Park 904, 1098 XH Amsterdam, The Netherlands.
authors.forename: Jan
authors.initials: J
authors.lastname: Gerretzen
authors.affiliation: Radboud University , Institute for Molecules and Materials, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands.|TI-COAST , Science Park 904, 1098 XH Amsterdam, The Netherlands.
authors.forename: Ewa
authors.initials: E
authors.lastname: Szymańska
authors.affiliation: Radboud University , Institute for Molecules and Materials, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands.
authors.forename: Jeroen J
authors.initials: JJ
authors.lastname: Jansen
authors.affiliation: AkzoNobel , Supply Chain, Research & Development, Zutphenseweg 10, 7418 AJ Deventer, The Netherlands.
authors.forename: Jacob
authors.initials: J
authors.lastname: Bart
authors.affiliation: AkzoNobel , Supply Chain, Research & Development, Zutphenseweg 10, 7418 AJ Deventer, The Netherlands.
authors.forename: Henk-Jan
authors.initials: HJ
authors.lastname: van Manen
authors.affiliation: Eindhoven University of Technology , Den Dolech 2, 5600 MB Eindhoven, The Netherlands.
authors.forename: Edwin R
authors.initials: ER
authors.lastname: van den Heuvel
authors.affiliation: Radboud University , Institute for Molecules and Materials, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands.
authors.forename: Lutgarde M C
authors.initials: LM
authors.lastname: Buydens