How many samples are needed to build a classifier: a general sequential approach.
BACKGROUND
The standard paradigm for a classifier design is to obtain a sample of feature-label pairs and then to apply a classification rule to derive a classifier from the sample data. Typically in laboratory situations the sample size is limited by cost, time or availability of sample material. Thus, an investigator may wish to consider a sequential approach in which there is a sufficient number of patients to train a classifier in order to make a sound decision for diagnosis while at the same time keeping the number of patients as small as possible to make the studies affordable.


RESULTS
A sequential classification procedure is studied via the martingale central limit theorem. It updates the classification rule at each step and provides stopping criteria to ensure with a certain confidence that at stopping a future subject will have misclassification probability smaller than a predetermined threshold. Simulation studies and applications to microarray data analysis are provided. The procedure possesses several attractive properties: (1) it updates the classification rule sequentially and thus does not rely on distributions of primary measurements from other studies; (2) it assesses the stopping criteria at each sequential step and thus can substantially reduce cost via early stopping; and (3) it is not restricted to any particular classification rule and therefore applies to any parametric or non-parametric method, including feature selection or extraction.


BACKGROUND
R-code for the sequential stopping rule is available at http://stat.tamu.edu/~wfu/microarray/sequential/R-code.html
Department of Statistics, Texas A&M University, 447 Blocker Building, College Station, TX 77843, USA. wfu@stat.tamu.edu
Wenjiang J
WJ
Fu
Edward R
ER
Dougherty
Bani
B
Mallick
Raymond J
RJ
Carroll
D000465:Algorithms; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000378:metabolism; D016000:Cluster Analysis; D006801:Humans; D015233:Models, Statistical; D009363:Neoplasm Proteins / Q000235:genetics; D020411:Oligonucleotide Array Sequence Analysis / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D018401:Sample Size; D012680:Sensitivity and Specificity; D017422:Sequence Analysis, DNA / Q000379:methods*
D009363:Neoplasm Proteins