title: Independent validation of a model using cell line chemosensitivity to predict response to therapy.
abstract: BACKGROUND
  Methods using cell line microarray and drug sensitivity data to predict patients' chemotherapy response are appealing, but groups may be reluctant to release details to preserve intellectual property. Here we describe a case study to validate predictions while treating the methods as a "black box."
  
  
  METHODS
  Medical Prognosis Institute (MPI) constructed cell-line-derived sensitivity scores (SSs) and combined scores (CSs) that incorporate clinical variables. MD Anderson researchers evaluated their predictions. We searched the Gene Expression Omnibus (GEO) to identify validation datasets, and we performed statistical evaluation of the agreement between prediction and clinical observation.
  
  
  RESULTS
  We identified 3 suitable datasets: GSE16446 (n = 120; binary outcome), GSE17920 (n = 130; binary outcome), and GSE10255 (n = 161; continuous and time-to-event outcomes). The SS was statistically significantly associated with primary treatment responses for all studies (GSE16446: P = .02; GSE17920: P = .02; GSE10255: P = .02). Dichotomized SSs performed no better than chance for GSE16446 and GSE17920, and categorized SSs did not predict disease-free survival (GSE10255). SSs sometimes improved on predictions using clinical variables (GSE16446: P = .05; GSE17920: P = .31; GSE10255: P = .045), but gains were limited (95% confidence intervals for GSE16446 and GSE17920 include 0). The CS did not predict treatment response for GSE16446 (P = .55), but it did for GSE17920 (P < .001). Coefficients of clinical variables provided by MPI for CSs agree with estimates for GSE17920 better than estimates for GSE16446.
  
  
  CONCLUSIONS
  Model predictions were better than chance in all three datasets. However, these scores added little to existing clinical predictors; statistically significant contributions were likely to be too small to change clinical practice. These findings suggest that discovering better predictors will require both cell line data and a clinical training dataset of patient samples.
authors: Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX 77230, USA.
authors: Wenting
authors: W
authors: Wang
authors: Keith A
authors: KA
authors: Baggerly
authors: Steen
authors: S
authors: Knudsen
authors: Jon
authors: J
authors: Askaa
authors: Wiktor
authors: W
authors: Mazin
authors: Kevin R
authors: KR
authors: Coombes
mesh_terms: D000328:Adult; D000368:Aged; D000970:Antineoplastic Agents / Q000494:pharmacology* / Q000627:therapeutic use; D000971:Antineoplastic Combined Chemotherapy Protocols / Q000008:administration & dosage; D019540:Area Under Curve; D001761:Bleomycin / Q000008:administration & dosage; D001943:Breast Neoplasms / Q000188:drug therapy*; D045744:Cell Line, Tumor / Q000187:drug effects*; D003606:Dacarbazine / Q000008:administration & dosage; D004317:Doxorubicin / Q000008:administration & dosage; D015251:Epirubicin / Q000008:administration & dosage; D005260:Female; D015972:Gene Expression Regulation, Neoplastic; D006689:Hodgkin Disease / Q000188:drug therapy*; D006801:Humans; D016014:Linear Models; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D015999:Multivariate Analysis; D054198:Precursor Cell Lymphoblastic Leukemia-Lymphoma / Q000188:drug therapy*; D011237:Predictive Value of Tests; D012372:ROC Curve; D016896:Treatment Outcome; D014747:Vinblastine / Q000008:administration & dosage
chemical_list: D000970:Antineoplastic Agents; D001761:Bleomycin; D015251:Epirubicin; D014747:Vinblastine; D003606:Dacarbazine; D004317:Doxorubicin
references: Nat Med. 2010 Feb;16(2):214-8
references: 20098429
references: Clin Chem. 2011 May;57(5):688-90
references: 21364027
references: Clin Cancer Res. 2009 Jan 15;15(2):502-10
references: 19147755
references: Nature. 2010 Sep 23;467(7314):401
references: 20864982
references: Circulation. 2007 Feb 20;115(7):928-35
references: 17309939
references: Mol Cancer Ther. 2012 Jan;11(1):34-44
references: 22027696
references: Lancet Oncol. 2007 Dec;8(12):1071-1078
references: 18024211
references: Nat Med. 2006 Nov;12(11):1294-300
references: 17057710
references: Nat Genet. 2001 Dec;29(4):365-71
references: 11726920
references: N Engl J Med. 2010 Mar 11;362(10):875-85
references: 20220182
references: Nat Med. 2007 Nov;13(11):1276-7; author reply 1277-8
references: 17987014
references: Nature. 2012 Mar 28;483(7391):603-7
references: 22460905
references: Biostatistics. 2007 Jan;8(1):118-27
references: 16632515
references: Lancet Oncol. 2010 Apr;11(4):358-65
references: 20189874
references: Stat Med. 2008 Jan 30;27(2):157-72; discussion 207-12
references: 17569110
references: Biom J. 2008 Aug;50(4):457-79
references: 18663757
references: Blood. 2002 Aug 15;100(4):1240-7
references: 12149204
references: N Engl J Med. 1998 Nov 19;339(21):1506-14
references: 9819449
references: PLoS Med. 2008 Apr 15;5(4):e83
references: 18416598
references: J Clin Oncol. 2005 Nov 20;23(33):8331-9
references: 16293864
references: J Clin Oncol. 2007 Oct 1;25(28):4350-7
references: 17906199
references: Epidemiology. 2010 Jan;21(1):128-38
references: 20010215