Comparison of 10 brain tissue segmentation methods using revisited IBSR annotations.
OBJECTIVE
Ground-truth annotations from the well-known Internet Brain Segmentation Repository (IBSR) datasets consider Sulcal cerebrospinal fluid (SCSF) voxels as gray matter. This can lead to bias when evaluating the performance of tissue segmentation methods. In this work we compare the accuracy of 10 brain tissue segmentation methods analyzing the effects of SCSF ground-truth voxels on accuracy estimations.


METHODS
The set of methods is composed by FAST, SPM5, SPM8, GAMIXTURE, ANN, FCM, KNN, SVPASEG, FANTASM, and PVC. Methods are evaluated using original IBSR ground-truth and ranked by means of their performance on pairwise comparisons using permutation tests. Afterward, the evaluation is repeated using IBSR ground-truth without considering SCSF.


RESULTS
The Dice coefficient of all methods is affected by changes in SCSF annotations, especially on SPM5, SPM8 and FAST. When not considering SCSF voxels, SVPASEG (0.90 ± 0.01) and SPM8 (0.91 ± 0.01) are the methods from our study that appear more suitable for gray matter tissue segmentation, while FAST (0.89 ± 0.02) is the best tool for segmenting white matter tissue.


CONCLUSIONS
The performance and the accuracy of methods on IBSR images vary notably when not considering SCSF voxels. The fact that three of the most common methods (FAST, SPM5, and SPM8) report an important change in their accuracy suggest to consider these differences in labeling for new comparative studies.
Department of Computer Architecture and Technology, University of Girona, Girona, (Spain).
Sergi
S
Valverde
Arnau
A
Oliver
Mariano
M
Cabezas
Eloy
E
Roura
Xavier
X
Lladó
D000465:Algorithms; D001921:Brain / Q000033:anatomy & histology*; D001931:Brain Mapping / Q000379:methods; D066264:Datasets as Topic / Q000706:statistics & numerical data*; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D008279:Magnetic Resonance Imaging / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results
IBSR; brain MRI; permutation tests; tissue segmentation