Test-retest and between-site reliability in a multicenter fMRI study.
In the present report, estimates of test-retest and between-site reliability of fMRI assessments were produced in the context of a multicenter fMRI reliability study (FBIRN Phase 1, www.nbirn.net). Five subjects were scanned on 10 MRI scanners on two occasions. The fMRI task was a simple block design sensorimotor task. The impulse response functions to the stimulation block were derived using an FIR-deconvolution analysis with FMRISTAT. Six functionally-derived ROIs covering the visual, auditory and motor cortices, created from a prior analysis, were used. Two dependent variables were compared: percent signal change and contrast-to-noise-ratio. Reliability was assessed with intraclass correlation coefficients derived from a variance components analysis. Test-retest reliability was high, but initially, between-site reliability was low, indicating a strong contribution from site and site-by-subject variance. However, a number of factors that can markedly improve between-site reliability were uncovered, including increasing the size of the ROIs, adjusting for smoothness differences, and inclusion of additional runs. By employing multiple steps, between-site reliability for 3T scanners was increased by 123%. Dropping one site at a time and assessing reliability can be a useful method of assessing the sensitivity of the results to particular sites. These findings should provide guidance toothers on the best practices for future multicenter studies.
Friedman
Lee
L
Department of Psychiatry and Human Behavior, University of California Irvine, Irvine, California, USA. lfriedman10@comcast.net
Stern
Hal
H
Brown
Gregory G
GG
Mathalon
Daniel H
DH
Turner
Jessica
J
Glover
Gary H
GH
Gollub
Randy L
RL
Lauriello
John
J
Lim
Kelvin O
KO
Cannon
Tyrone
T
Greve
Douglas N
DN
Bockholt
Henry Jeremy
HJ
Belger
Aysenil
A
Mueller
Bryon
B
Doty
Michael J
MJ
He
Jianchun
J
Wells
William
W
Smyth
Padhraic
P
Pieper
Steve
S
Kim
Seyoung
S
Kubicki
Marek
M
Vangel
Mark
M
Potkin
Steven G
SG
D000328:Adult; D006801:Humans; D008279:Magnetic Resonance Imaging / Q000592:standards*; D008297:Male; D015337:Multicenter Studies as Topic / Q000379:methods / Q000592:standards*; D012107:Research Design / Q000592:standards*