Variability in human and automatic segmentation of melanocytic lesions.
In a double blind evaluation of 60 digital dermatoscopic images by 4 "junior", 4 "senior" and 4 "expert" dermatologists (dermatoscopy training respectively less than 1 year, between 1 and 5 years, and more than 5 years), a significant inter-operator variability was observed in melanocytic lesion border identification (with a disagreement of the order of 10 - 20% of the area of the lesions). Expert dermatologists showed greater agreement among themselves than with senior and junior dermatologists, and a slight tendency towards "tighter" segmentations. The human inter-operator variability was then used to evaluate the segmentation accuracy of 4 algorithms, representative of the 3 fundamental state-of-the-art automated segmentation techniques and of a fourth, novel, technique. Our evaluation methodology addresses a number of crucial difficulties encountered in previous studies and may be of independent interest. 3 of the 4 algorithms showed considerably less agreement with expert dermatologists than even senior and junior dermatologists did (with a disagreement of the order of 30% of the area of the lesions); the remaining algorithm, however, showed agreement with expert dermatologists comparable to that of other expert dermatologists.
Silletti
A
A
Department of Information Engineering, University of Padova, Italy. melanoma@dei.unipd.it
Peserico
E
E
Mantovan
A
A
Zattra
E
E
Peserico
A
A
Belloni Fortina
A
A
D000465:Algorithms; D001185:Artificial Intelligence; D046169:Dermoscopy / Q000379:methods*; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D009508:Nevus, Pigmented / Q000473:pathology*; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity