Multi-modality CADx: ROC study of the effect on radiologists' accuracy in characterizing breast masses on mammograms and 3D ultrasound images.
OBJECTIVE
To investigate the effect of a computer-aided diagnosis (CADx) system on radiologists' performance in discriminating malignant and benign masses on mammograms and three-dimensional (3D) ultrasound (US) images.


METHODS
Our dataset contained mammograms and 3D US volumes from 67 women (median age, 51; range: 27-86) with 67 biopsy-proven breast masses (32 benign and 35 malignant). A CADx system was designed to automatically delineate the mass boundaries on mammograms and the US volumes, extract features, and merge the extracted features into a multi-modality malignancy score. Ten experienced readers (subspecialty academic breast imaging radiologists) first viewed the mammograms alone, and provided likelihood of malignancy (LM) ratings and Breast Imaging and Reporting System assessments. Subsequently, the reader viewed the US images with the mammograms, and provided LM and action category ratings. Finally, the CADx score was shown and the reader had the opportunity to revise the ratings. The LM ratings were analyzed using receiver-operating characteristic (ROC) methodology, and the action category ratings were used to determine the sensitivity and specificity of cancer diagnosis.


RESULTS
Without CADx, readers' average area under the ROC curve, A(z), was 0.93 (range, 0.86-0.96) for combined assessment of the mass on both the US volume and mammograms. With CADx, their average A(z) increased to 0.95 (range, 0.91-0.98), which was borderline significant (P = .05). The average sensitivity of the readers increased from 98% to 99% with CADx, while the average specificity increased from 27% to 29%. The change in sensitivity with CADx did not achieve statistical significance for the individual radiologists, and the change in specificity was statistically significant for one of the radiologists.


CONCLUSIONS
A well-trained CADx system that combines features extracted from mammograms and US images may have the potential to improve radiologists' performance in distinguishing malignant from benign breast masses and making decisions about biopsies.
Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109-5842, USA.
Berkman
B
Sahiner
Heang-Ping
HP
Chan
Lubomir M
LM
Hadjiiski
Marilyn A
MA
Roubidoux
Chintana
C
Paramagul
Janet E
JE
Bailey
Alexis V
AV
Nees
Caroline E
CE
Blane
Dorit D
DD
Adler
Stephanie K
SK
Patterson
Katherine A
KA
Klein
Renee W
RW
Pinsky
Mark A
MA
Helvie
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D005260:Female; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000379:methods*; D008327:Mammography / Q000379:methods*; D008875:Middle Aged; D015588:Observer Variation; D012372:ROC Curve; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013382:Subtraction Technique; D014463:Ultrasonography