The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models.
Gene expression data from microarrays are being applied to predict preclinical and clinical endpoints, but the reliability of these predictions has not been established. In the MAQC-II project, 36 independent teams analyzed six microarray data sets to generate predictive models for classifying a sample with respect to one of 13 endpoints indicative of lung or liver toxicity in rodents, or of breast cancer, multiple myeloma or neuroblastoma in humans. In total, >30,000 models were built using many combinations of analytical methods. The teams generated predictive models without knowing the biological meaning of some of the endpoints and, to mimic clinical reality, tested the models on data that had not been used for training. We found that model performance depended largely on the endpoint and team proficiency and that different approaches generated models of similar performance. The conclusions and recommendations from MAQC-II should be useful for regulatory agencies, study committees and independent investigators that evaluate methods for global gene expression analysis.
Shi
Leming
L
National Center for Toxicological Research, US Food and Drug Administration, Jefferson, Arkansas, USA.
Campbell
Gregory
G
Jones
Wendell D
WD
Campagne
Fabien
F
Wen
Zhining
Z
Walker
Stephen J
SJ
Su
Zhenqiang
Z
Chu
Tzu-Ming
TM
Goodsaid
Federico M
FM
Pusztai
Lajos
L
Shaughnessy
John D
JD
Oberthuer
André
A
Thomas
Russell S
RS
Paules
Richard S
RS
Fielden
Mark
M
Barlogie
Bart
B
Chen
Weijie
W
Du
Pan
P
Fischer
Matthias
M
Furlanello
Cesare
C
Gallas
Brandon D
BD
Ge
Xijin
X
Megherbi
Dalila B
DB
Symmans
W Fraser
WF
Wang
May D
MD
Zhang
John
J
Bitter
Hans
H
Brors
Benedikt
B
Bushel
Pierre R
PR
Bylesjo
Max
M
Chen
Minjun
M
Cheng
Jie
J
Cheng
Jing
J
Chou
Jeff
J
Davison
Timothy S
TS
Delorenzi
Mauro
M
Deng
Youping
Y
Devanarayan
Viswanath
V
Dix
David J
DJ
Dopazo
Joaquin
J
Dorff
Kevin C
KC
Elloumi
Fathi
F
Fan
Jianqing
J
Fan
Shicai
S
Fan
Xiaohui
X
Fang
Hong
H
Gonzaludo
Nina
N
Hess
Kenneth R
KR
Hong
Huixiao
H
Huan
Jun
J
Irizarry
Rafael A
RA
Judson
Richard
R
Juraeva
Dilafruz
D
Lababidi
Samir
S
Lambert
Christophe G
CG
Li
Li
L
Li
Yanen
Y
Li
Zhen
Z
Lin
Simon M
SM
Liu
Guozhen
G
Lobenhofer
Edward K
EK
Luo
Jun
J
Luo
Wen
W
McCall
Matthew N
MN
Nikolsky
Yuri
Y
Pennello
Gene A
GA
Perkins
Roger G
RG
Philip
Reena
R
Popovici
Vlad
V
Price
Nathan D
ND
Qian
Feng
F
Scherer
Andreas
A
Shi
Tieliu
T
Shi
Weiwei
W
Sung
Jaeyun
J
Thierry-Mieg
Danielle
D
Thierry-Mieg
Jean
J
Thodima
Venkata
V
Trygg
Johan
J
Vishnuvajjala
Lakshmi
L
Wang
Sue Jane
SJ
Wu
Jianping
J
Wu
Yichao
Y
Xie
Qian
Q
Yousef
Waleed A
WA
Zhang
Liang
L
Zhang
Xuegong
X
Zhong
Sheng
S
Zhou
Yiming
Y
Zhu
Sheng
S
Arasappan
Dhivya
D
Bao
Wenjun
W
Lucas
Anne Bergstrom
AB
Berthold
Frank
F
Brennan
Richard J
RJ
Buness
Andreas
A
Catalano
Jennifer G
JG
Chang
Chang
C
Chen
Rong
R
Cheng
Yiyu
Y
Cui
Jian
J
Czika
Wendy
W
Demichelis
Francesca
F
Deng
Xutao
X
Dosymbekov
Damir
D
Eils
Roland
R
Feng
Yang
Y
Fostel
Jennifer
J
Fulmer-Smentek
Stephanie
S
Fuscoe
James C
JC
Gatto
Laurent
L
Ge
Weigong
W
Goldstein
Darlene R
DR
Guo
Li
L
Halbert
Donald N
DN
Han
Jing
J
Harris
Stephen C
SC
Hatzis
Christos
C
Herman
Damir
D
Huang
Jianping
J
Jensen
Roderick V
RV
Jiang
Rui
R
Johnson
Charles D
CD
Jurman
Giuseppe
G
Kahlert
Yvonne
Y
Khuder
Sadik A
SA
Kohl
Matthias
M
Li
Jianying
J
Li
Li
L
Li
Menglong
M
Li
Quan-Zhen
QZ
Li
Shao
S
Li
Zhiguang
Z
Liu
Jie
J
Liu
Ying
Y
Liu
Zhichao
Z
Meng
Lu
L
Madera
Manuel
M
Martinez-Murillo
Francisco
F
Medina
Ignacio
I
Meehan
Joseph
J
Miclaus
Kelci
K
Moffitt
Richard A
RA
Montaner
David
D
Mukherjee
Piali
P
Mulligan
George J
GJ
Neville
Padraic
P
Nikolskaya
Tatiana
T
Ning
Baitang
B
Page
Grier P
GP
Parker
Joel
J
Parry
R Mitchell
RM
Peng
Xuejun
X
Peterson
Ron L
RL
Phan
John H
JH
Quanz
Brian
B
Ren
Yi
Y
Riccadonna
Samantha
S
Roter
Alan H
AH
Samuelson
Frank W
FW
Schumacher
Martin M
MM
Shambaugh
Joseph D
JD
Shi
Qiang
Q
Shippy
Richard
R
Si
Shengzhu
S
Smalter
Aaron
A
Sotiriou
Christos
C
Soukup
Mat
M
Staedtler
Frank
F
Steiner
Guido
G
Stokes
Todd H
TH
Sun
Qinglan
Q
Tan
Pei-Yi
PY
Tang
Rong
R
Tezak
Zivana
Z
Thorn
Brett
B
Tsyganova
Marina
M
Turpaz
Yaron
Y
Vega
Silvia C
SC
Visintainer
Roberto
R
von Frese
Juergen
J
Wang
Charles
C
Wang
Eric
E
Wang
Junwei
J
Wang
Wei
W
Westermann
Frank
F
Willey
James C
JC
Woods
Matthew
M
Wu
Shujian
S
Xiao
Nianqing
N
Xu
Joshua
J
Xu
Lei
L
Yang
Lun
L
Zeng
Xiao
X
Zhang
Jialu
J
Zhang
Li
L
Zhang
Min
M
Zhao
Chen
C
Puri
Raj K
RK
Scherf
Uwe
U
Tong
Weida
W
Wolfinger
Russell D
RD
D000818:Animals; D001943:Breast Neoplasms / Q000175:diagnosis / Q000235:genetics; D004195:Disease Models, Animal; D005260:Female; D020869:Gene Expression Profiling / Q000379:methods / Q000592:standards; D017408:Guidelines as Topic; D006801:Humans; D008107:Liver Diseases / Q000209:etiology / Q000235:genetics* / Q000473:pathology; D008171:Lung Diseases / Q000209:etiology / Q000235:genetics* / Q000473:pathology; D009101:Multiple Myeloma / Q000175:diagnosis / Q000235:genetics; D009369:Neoplasms / Q000175:diagnosis / Q000235:genetics* / Q000401:mortality*; D009447:Neuroblastoma / Q000175:diagnosis / Q000235:genetics; D020411:Oligonucleotide Array Sequence Analysis / Q000379:methods* / Q000592:standards*; D011237:Predictive Value of Tests; D011786:Quality Control; D051381:Rats; D016019:Survival Analysis