title: A protocol for building and evaluating predictors of disease state based on microarray data.
abstract: BACKGROUND
  Microarray gene expression data are increasingly employed to identify sets of marker genes that accurately predict disease development and outcome in cancer. Many computational approaches have been proposed to construct such predictors. However, there is, as yet, no objective way to evaluate whether a new approach truly improves on the current state of the art. In addition no 'standard' computational approach has emerged which enables robust outcome prediction.
  
  
  RESULTS
  An important contribution of this work is the description of a principled training and validation protocol, which allows objective evaluation of the complete methodology for constructing a predictor. We review the possible choices of computational approaches, with specific emphasis on predictor choice and reporter selection strategies. Employing this training-validation protocol, we evaluated different reporter selection strategies and predictors on six gene expression datasets of varying degrees of difficulty. We demonstrate that simple reporter selection strategies (forward filtering and shrunken centroids) work surprisingly well and outperform partial least squares in four of the six datasets. Similarly, simple predictors, such as the nearest mean classifier, outperform more complex classifiers. Our training-validation protocol provides a robust methodology to evaluate the performance of new computational approaches and to objectively compare outcome predictions on different datasets.
authors.affiliation: Department of Mediamatics, Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology Mekelweg 4, 2628 CD Delft, The Netherlands. l.f.a.wessels@ewi.tudelft.nl
authors.forename: Lodewyk F A
authors.initials: LF
authors.lastname: Wessels
authors.forename: Marcel J T
authors.initials: MJ
authors.lastname: Reinders
authors.forename: Augustinus A M
authors.initials: AA
authors.lastname: Hart
authors.forename: Cor J
authors.initials: CJ
authors.lastname: Veenman
authors.forename: Hongyue
authors.initials: H
authors.lastname: Dai
authors.forename: Yudong D
authors.initials: YD
authors.lastname: He
authors.forename: Laura J
authors.initials: LJ
authors.lastname: van't Veer
mesh_terms: D000465:Algorithms; D000818:Animals; D014408:Biomarkers, Tumor / Q000032:analysis*; D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D005246:Feedback; D020869:Gene Expression Profiling / Q000379:methods*; D006801:Humans; D009363:Neoplasm Proteins / Q000032:analysis*; D009369:Neoplasms / Q000175:diagnosis* / Q000378:metabolism*; D020411:Oligonucleotide Array Sequence Analysis / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012986:Software Validation
chemical_list: D014408:Biomarkers, Tumor; D009363:Neoplasm Proteins