title: Prediction errors in learning drug response from gene expression data - influence of labeling, sample size, and machine learning algorithm.
abstract: Model-based prediction is dependent on many choices ranging from the sample collection and prediction endpoint to the choice of algorithm and its parameters. Here we studied the effects of such choices, exemplified by predicting sensitivity (as IC50) of cancer cell lines towards a variety of compounds. For this, we used three independent sample collections and applied several machine learning algorithms for predicting a variety of endpoints for drug response. We compared all possible models for combinations of sample collections, algorithm, drug, and labeling to an identically generated null model. The predictability of treatment effects varies among compounds, i.e. response could be predicted for some but not for all. The choice of sample collection plays a major role towards lowering the prediction error, as does sample size. However, we found that no algorithm was able to consistently outperform the other and there was no significant difference between regression and two- or three class predictors in this experimental setting. These results indicate that response-modeling projects should direct efforts mainly towards sample collection and data quality, rather than method adjustment.
authors.affiliation: Aachen Institute for Advanced Study in Computational Engineering Science (AICES), RWTH Aachen University, Aachen, Germany.
authors.forename: Immanuel
authors.initials: I
authors.lastname: Bayer
authors.forename: Philip
authors.initials: P
authors.lastname: Groth
authors.forename: Sebastian
authors.initials: S
authors.lastname: Schneckener
mesh_terms: D000465:Algorithms; D000970:Antineoplastic Agents / Q000494:pharmacology*; D001185:Artificial Intelligence / Q000592:standards*; D045744:Cell Line, Tumor; D049109:Cell Proliferation / Q000187:drug effects; D005544:Forecasting / Q000379:methods*; D015870:Gene Expression / Q000187:drug effects*; D006801:Humans; D020128:Inhibitory Concentration 50; D046228:Microarray Analysis; D008954:Models, Biological; D009369:Neoplasms / Q000188:drug therapy; D010363:Pattern Recognition, Automated / Q000592:standards*; D018401:Sample Size
chemical_list: D000970:Antineoplastic Agents
references.citation: Nucleic Acids Res. 2012 Jan;40(Database issue):D13-25
references.pmid: 22140104
references.citation: J Stat Softw. 2010;33(1):1-22
references.pmid: 20808728
references.citation: Eur J Cancer. 2004 Apr;40(6):785-93
references.pmid: 15120034
references.citation: BMC Bioinformatics. 2006 Jan 06;7:3
references.pmid: 16398926
references.citation: Nat Biotechnol. 2010 Aug;28(8):827-38
references.pmid: 20676074
references.citation: Proc Natl Acad Sci U S A. 2000 Jan 4;97(1):262-7
references.pmid: 10618406
references.citation: Cancer Res. 2012 Jul 15;72(14):3499-511
references.pmid: 22802077
references.citation: Bioinformatics. 2011 Jan 15;27(2):220-4
references.pmid: 21134890
references.citation: BMC Med Genomics. 2011 Oct 14;4:73
references.pmid: 21996057
references.citation: BMC Bioinformatics. 2008 Jul 22;9:319
references.pmid: 18647401
references.citation: Bioinformatics. 2002 Dec;18(12):1585-92
references.pmid: 12490442
references.citation: Biostatistics. 2004 Jul;5(3):427-43
references.pmid: 15208204
references.citation: Nucleic Acids Res. 2009 Jan;37(Database issue):D868-72
references.pmid: 19015125