Multi-factorial analysis of class prediction error: estimating optimal number of biomarkers for various classification rules.
Machine learning and statistical model based classifiers have increasingly been used with more complex and high dimensional biological data obtained from high-throughput technologies. Understanding the impact of various factors associated with large and complex microarray datasets on the predictive performance of classifiers is computationally intensive, under investigated, yet vital in determining the optimal number of biomarkers for various classification purposes aimed towards improved detection, diagnosis, and therapeutic monitoring of diseases. We investigate the impact of microarray based data characteristics on the predictive performance for various classification rules using simulation studies. Our investigation using Random Forest, Support Vector Machines, Linear Discriminant Analysis and k-Nearest Neighbour shows that the predictive performance of classifiers is strongly influenced by training set size, biological and technical variability, replication, fold change and correlation between biomarkers. Optimal number of biomarkers for a classification problem should therefore be estimated taking account of the impact of all these factors. A database of average generalization errors is built for various combinations of these factors. The database of generalization errors can be used for estimating the optimal number of biomarkers for given levels of predictive accuracy as a function of these factors. Examples show that curves from actual biological data resemble that of simulated data with corresponding levels of data characteristics. An R package optBiomarker implementing the method is freely available for academic use from the Comprehensive R Archive Network (http://www.cran.r-project.org/web/packages/optBiomarker/).
Department of Biostatistics, Institute of Psychiatry and NIHR Biomedical, Research Centre for Mental Health at the South London and Maudsley NHS Foundation Trust, King's College London, De Crespigny Park, London, UK. mizanur.khondoker@kcl.ac.uk
Mizanur R
MR
Khondoker
Till T
TT
Bachmann
Muriel
M
Mewissen
Paul
P
Dickinson
Bartosz
B
Dobrzelecki
Colin J
CJ
Campbell
Andrew R
AR
Mount
Anthony J
AJ
Walton
Jason
J
Crain
Holger
H
Schulze
Gerard
G
Giraud
Alan J
AJ
Ross
Ilenia
I
Ciani
Stuart W J
SW
Ember
Chaker
C
Tlili
Jonathan G
JG
Terry
Eilidh
E
Grant
Nicola
N
McDonnell
Peter
P
Ghazal
D001185:Artificial Intelligence; D015415:Biomarkers / Q000097:blood; D002965:Classification / Q000379:methods; D019295:Computational Biology; D016208:Databases, Factual; D020869:Gene Expression Profiling / Q000706:statistics & numerical data; D006801:Humans; D046228:Microarray Analysis / Q000706:statistics & numerical data; D015233:Models, Statistical; D020411:Oligonucleotide Array Sequence Analysis / Q000706:statistics & numerical data
D015415:Biomarkers