title: Systematic analysis of challenge-driven improvements in molecular prognostic models for breast cancer.
abstract: Although molecular prognostics in breast cancer are among the most successful examples of translating genomic analysis to clinical applications, optimal approaches to breast cancer clinical risk prediction remain controversial. The Sage Bionetworks-DREAM Breast Cancer Prognosis Challenge (BCC) is a crowdsourced research study for breast cancer prognostic modeling using genome-scale data. The BCC provided a community of data analysts with a common platform for data access and blinded evaluation of model accuracy in predicting breast cancer survival on the basis of gene expression data, copy number data, and clinical covariates. This approach offered the opportunity to assess whether a crowdsourced community Challenge would generate models of breast cancer prognosis commensurate with or exceeding current best-in-class approaches. The BCC comprised multiple rounds of blinded evaluations on held-out portions of data on 1981 patients, resulting in more than 1400 models submitted as open source code. Participants then retrained their models on the full data set of 1981 samples and submitted up to five models for validation in a newly generated data set of 184 breast cancer patients. Analysis of the BCC results suggests that the best-performing modeling strategy outperformed previously reported methods in blinded evaluations; model performance was consistent across several independent evaluations; and aggregating community-developed models achieved performance on par with the best-performing individual models.
authors.affiliation: Sage Bionetworks, 1100 Fairview Avenue North, MS: M1-C108, Seattle, WA 98109, USA. margolin@sagebase.org
authors.forename: Adam A
authors.initials: AA
authors.lastname: Margolin
authors.forename: Erhan
authors.initials: E
authors.lastname: Bilal
authors.forename: Erich
authors.initials: E
authors.lastname: Huang
authors.forename: Thea C
authors.initials: TC
authors.lastname: Norman
authors.forename: Lars
authors.initials: L
authors.lastname: Ottestad
authors.forename: Brigham H
authors.initials: BH
authors.lastname: Mecham
authors.forename: Ben
authors.initials: B
authors.lastname: Sauerwine
authors.forename: Michael R
authors.initials: MR
authors.lastname: Kellen
authors.forename: Lara M
authors.initials: LM
authors.lastname: Mangravite
authors.forename: Matthew D
authors.initials: MD
authors.lastname: Furia
authors.forename: Hans Kristian Moen
authors.initials: HK
authors.lastname: Vollan
authors.forename: Oscar M
authors.initials: OM
authors.lastname: Rueda
authors.forename: Justin
authors.initials: J
authors.lastname: Guinney
authors.forename: Nicole A
authors.initials: NA
authors.lastname: Deflaux
authors.forename: Bruce
authors.initials: B
authors.lastname: Hoff
authors.forename: Xavier
authors.initials: X
authors.lastname: Schildwachter
authors.forename: Hege G
authors.initials: HG
authors.lastname: Russnes
authors.forename: Daehoon
authors.initials: D
authors.lastname: Park
authors.forename: Veronica O
authors.initials: VO
authors.lastname: Vang
authors.forename: Tyler
authors.initials: T
authors.lastname: Pirtle
authors.forename: Lamia
authors.initials: L
authors.lastname: Youseff
authors.forename: Craig
authors.initials: C
authors.lastname: Citro
authors.forename: Christina
authors.initials: C
authors.lastname: Curtis
authors.forename: Vessela N
authors.initials: VN
authors.lastname: Kristensen
authors.forename: Joseph
authors.initials: J
authors.lastname: Hellerstein
authors.forename: Stephen H
authors.initials: SH
authors.lastname: Friend
authors.forename: Gustavo
authors.initials: G
authors.lastname: Stolovitzky
authors.forename: Samuel
authors.initials: S
authors.lastname: Aparicio
authors.forename: Carlos
authors.initials: C
authors.lastname: Caldas
authors.forename: Anne-Lise
authors.initials: AL
authors.lastname: Børresen-Dale
mesh_terms: D001943:Breast Neoplasms / Q000175:diagnosis* / Q000235:genetics*; D030541:Databases, Genetic; D005260:Female; D006801:Humans; D008875:Middle Aged; D008954:Models, Biological; D011379:Prognosis; D016019:Survival Analysis; D013997:Time Factors