title: LUNGx Challenge for computerized lung nodule classification.
abstract: The purpose of this work is to describe the LUNGx Challenge for the computerized classification of lung nodules on diagnostic computed tomography (CT) scans as benign or malignant and report the performance of participants' computerized methods along with that of six radiologists who participated in an observer study performing the same Challenge task on the same dataset. The Challenge provided sets of calibration and testing scans, established a performance assessment process, and created an infrastructure for case dissemination and result submission. Ten groups applied their own methods to 73 lung nodules (37 benign and 36 malignant) that were selected to achieve approximate size matching between the two cohorts. Area under the receiver operating characteristic curve (AUC) values for these methods ranged from 0.50 to 0.68; only three methods performed statistically better than random guessing. The radiologists' AUC values ranged from 0.70 to 0.85; three radiologists performed statistically better than the best-performing computer method. The LUNGx Challenge compared the performance of computerized methods in the task of differentiating benign from malignant lung nodules on CT scans, placed in the context of the performance of radiologists on the same task. The continued public availability of the Challenge cases will provide a valuable resource for the medical imaging research community.
authors.affiliation: The University of Chicago , Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States.
authors.forename: Samuel G
authors.initials: SG
authors.lastname: Armato
authors.affiliation: The University of Chicago , Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States.
authors.forename: Karen
authors.initials: K
authors.lastname: Drukker
authors.affiliation: The University of Chicago , Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States.
authors.forename: Feng
authors.initials: F
authors.lastname: Li
authors.affiliation: University of Michigan , Department of Radiology, 1500 East Medical Center Drive, Ann Arbor, Michigan 48109, United States.
authors.forename: Lubomir
authors.initials: L
authors.lastname: Hadjiiski
authors.affiliation: Health Data Sciences Institute , Biomedical Science and Engineering Center, Oak Ridge National Laboratory, P.O. Box 2008 MS6085 Oak Ridge, Tennessee 37831-6085, United States.
authors.forename: Georgia D
authors.initials: GD
authors.lastname: Tourassi
authors.affiliation: The University of Chicago , Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States.
authors.forename: Roger M
authors.initials: RM
authors.lastname: Engelmann
authors.affiliation: The University of Chicago , Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States.
authors.forename: Maryellen L
authors.initials: ML
authors.lastname: Giger
authors.affiliation: National Cancer Institute , Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States.
authors.forename: George
authors.initials: G
authors.lastname: Redmond
authors.affiliation: National Cancer Institute , Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States.
authors.forename: Keyvan
authors.initials: K
authors.lastname: Farahani
authors.affiliation: Leidos Biomedical Research, Inc. , Frederick National Laboratory for Cancer Research, Cancer Imaging Program, 8560 Progress Drive, Frederick, Maryland 21702, United States.
authors.forename: Justin S
authors.initials: JS
authors.lastname: Kirby
authors.affiliation: National Cancer Institute , Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States.
authors.forename: Laurence P
authors.initials: LP
authors.lastname: Clarke
keywords: challenge; classification; computed tomography; computer-aided diagnosis; image analysis; lung nodule