Heat maps: an aid for data analysis and understanding of ROC CAD experiments.
OBJECTIVE
To demonstrate the value of a new data visualization and exploration method for mutlireader-multicase receiver operating characteristic (MRMC-ROC) experiments of computer-aided detection (CAD) algorithms that uses three-dimensional (3D) heat maps tool adapted from gene expression array analysis.


METHODS
We are using data from a clinical trial of a commercial CAD system for lung cancer detection (RapidScreen RS-2000, Riverain Medical Group, Miamisburg, OH, and Rockville, MD). 3D heat maps, originally developed for displaying changes in gene expression after cancer chemotherapy in MATLAB, were modified to display the radiologists confidence levels as they interpreted chest radiographs and used to visualize the radiologists confidence levels before and after the provision of a CAD system.


RESULTS
Heat maps demonstrated the variation among radiologists in their interpretation, and the degree of variation in interpretation when a single radiologist reinterpreted the same case without and with CAD modality. They demonstrated the variability in the identification of each cancer/cancer-free case and the variability of change seen when CAD prompts were provided.


CONCLUSIONS
CAD increases the consistency of interpretation of a single radiologist and of a group of radiologists. Heat maps provide a method for data visualization that clarifies the effects of reader variability in ROC CAD experiments. We demonstrated how heat maps can be used to document the complexity of reader variability and suggested how clustering can reveal both nonintuitive and intuitive groupings of cases, readers, and the interaction of both with CAD.
Freedman
Matthew T
MT
Division of Cancer Genetics and Epidemiology, Department of Oncology, Georgetown University Medical Center, Lombardi Building S150 (Office: S184A), Washington, DC 20057, USA. freedmmt@georgetown.edu
Osicka
Teresa
T
D000465:Algorithms; D002289:Carcinoma, Non-Small-Cell Lung / Q000000981:diagnostic imaging*; D003116:Color; D003936:Diagnosis, Computer-Assisted; D006358:Hot Temperature; D006801:Humans; D021621:Imaging, Three-Dimensional; D008175:Lung Neoplasms / Q000000981:diagnostic imaging*; D015588:Observer Variation; D012372:ROC Curve; D013902:Radiography, Thoracic