title: Generalized overlap measures for evaluation and validation in medical image analysis.
abstract: Measures of overlap of labelled regions of images, such as the Dice and Tanimoto coefficients, have been extensively used to evaluate image registration and segmentation algorithms. Modern studies can include multiple labels defined on multiple images yet most evaluation schemes report one overlap per labelled region, simply averaged over multiple images. In this paper, common overlap measures are generalized to measure the total overlap of ensembles of labels defined on multiple test images and account for fractional labels using fuzzy set theory. This framework allows a single "figure-of-merit" to be reported which summarises the results of a complex experiment by image pair, by label or overall. A complementary measure of error, the overlap distance, is defined which captures the spatial extent of the nonoverlapping part and is related to the Hausdorff distance computed on grey level images. The generalized overlap measures are validated on synthetic images for which the overlap can be computed analytically and used as similarity measures in nonrigid registration of three-dimensional magnetic resonance imaging (MRI) brain images. Finally, a pragmatic segmentation ground truth is constructed by registering a magnetic resonance atlas brain to 20 individual scans, and used with the overlap measures to evaluate publicly available brain segmentation algorithms.
authors.affiliation: Center for Medical Image Computing, University College London, London WC1E 6BT, UK. b.crum@ucl.ac.uk
authors.forename: William R
authors.initials: WR
authors.lastname: Crum
authors.forename: Oscar
authors.initials: O
authors.lastname: Camara
authors.forename: Derek L G
authors.initials: DL
authors.lastname: Hill
mesh_terms: D000465:Algorithms; D001921:Brain / Q000033:anatomy & histology*; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods; D008279:Magnetic Resonance Imaging / Q000379:methods* / Q000592:standards; D010363:Pattern Recognition, Automated / Q000379:methods* / Q000592:standards; D011785:Quality Assurance, Health Care / Q000379:methods / Q000592:standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013382:Subtraction Technique