title: A general approach to simultaneous model fitting and variable elimination in response models for biological data with many more variables than observations.
abstract: BACKGROUND
  With the advent of high throughput biotechnology data acquisition platforms such as micro arrays, SNP chips and mass spectrometers, data sets with many more variables than observations are now routinely being collected. Finding relationships between response variables of interest and variables in such data sets is an important problem akin to finding needles in a haystack. Whilst methods for a number of response types have been developed a general approach has been lacking.
  
  
  RESULTS
  The major contribution of this paper is to present a unified methodology which allows many common (statistical) response models to be fitted to such data sets. The class of models includes virtually any model with a linear predictor in it, for example (but not limited to), multiclass logistic regression (classification), generalised linear models (regression) and survival models. A fast algorithm for finding sparse well fitting models is presented. The ideas are illustrated on real data sets with numbers of variables ranging from thousands to millions. R code implementing the ideas is available for download.
  
  
  CONCLUSIONS
  The method described in this paper enables existing work on response models when there are less variables than observations to be leveraged to the situation when there are many more variables than observations. It is a powerful approach to finding parsimonious models for such datasets. The method is capable of handling problems with millions of variables and a large variety of response types within the one framework. The method compares favourably to existing methods such as support vector machines and random forests, but has the advantage of not requiring separate variable selection steps. It is also works for data types which these methods were not designed to handle. The method usually produces very sparse models which make biological interpretation simpler and more focused.
authors.affiliation: CSIRO Mathematical and Information Sciences, The Leeuwin Center, 65 Brockway Road, Floreat, 6014, Western Australia. harri.kiiveri@csiro.au
authors.forename: Harri T
authors.initials: HT
authors.lastname: Kiiveri
mesh_terms: D005260:Female; D020869:Gene Expression Profiling / Q000379:methods; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D007938:Leukemia / Q000145:classification / Q000235:genetics / Q000473:pathology; D008223:Lymphoma / Q000235:genetics / Q000401:mortality; D008297:Male; D008954:Models, Biological; D015233:Models, Statistical; D009367:Neoplasm Staging / Q000379:methods; D020411:Oligonucleotide Array Sequence Analysis; D010363:Pattern Recognition, Automated / Q000379:methods; D011471:Prostatic Neoplasms / Q000235:genetics / Q000473:pathology; D012016:Reference Values; D012107:Research Design; D012907:Smoking / Q000235:genetics; D012984:Software