title: Algorithm Variability in the Estimation of Lung Nodule Volume From Phantom CT Scans: Results of the QIBA 3A Public Challenge.
abstract: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA).
  
  
  
  The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type.
  
  
  
  Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall.
  
  
  
  The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.
authors.affiliation: Definiens AG, Bernhard-Wicki Str 5, 80636 Munich, Germany. Electronic address: mathelogou@definiens.com.
authors.forename: Maria
authors.initials: M
authors.lastname: Athelogou
authors.affiliation: UCLA, Center for Computer Vision and Imaging Biomarkers, Dept. of Radiological Sciences David Geffen School of Medicine at UCLA Dept. of Biostatistics Fielding School of Public at UCLA, Los Angeles, USA.
authors.forename: Hyun J
authors.initials: HJ
authors.lastname: Kim
authors.affiliation: National Institute of Standards and Technology, Gaithersburg, USA.
authors.forename: Alden
authors.initials: A
authors.lastname: Dima
authors.affiliation: Quantitative Health Sciences/JJN3, Cleveland Clinic Foundation, Cleveland, USA.
authors.forename: Nancy
authors.initials: N
authors.lastname: Obuchowski
authors.affiliation: National Institute of Standards and Technology, Gaithersburg, USA.
authors.forename: Adele
authors.initials: A
authors.lastname: Peskin
authors.affiliation: U.S. Food and Drug Administration, Silver Spring, Maryland.
authors.forename: Marios A
authors.initials: MA
authors.lastname: Gavrielides
authors.affiliation: U.S. Food and Drug Administration, Silver Spring, Maryland.
authors.forename: Nicholas
authors.initials: N
authors.lastname: Petrick
authors.affiliation: National Institute of Standards and Technology, Gaithersburg, USA.
authors.forename: Ganesh
authors.initials: G
authors.lastname: Saiprasad
authors.affiliation: Consultant QM/RA, Jena, Germany.
authors.forename: Dirk
authors.initials: D
authors.lastname: Colditz Colditz
authors.affiliation: MEDIAN Technologies, Valbonne Sophia Antipolis, France.
authors.forename: Hubert
authors.initials: H
authors.lastname: Beaumont
authors.affiliation: MEDIAN Technologies, Valbonne Sophia Antipolis, France.
authors.forename: Estanislao
authors.initials: E
authors.lastname: Oubel
authors.affiliation: Columbia University Medical Center, Department of Radiology, New York, USA.
authors.forename: Yongqiang
authors.initials: Y
authors.lastname: Tan
authors.affiliation: Columbia University Medical Center, Department of Radiology, New York, USA.
authors.forename: Binsheng
authors.initials: B
authors.lastname: Zhao
authors.affiliation: Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany.
authors.forename: Jan-Martin
authors.initials: JM
authors.lastname: Kuhnigk
authors.affiliation: Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany.
authors.forename: Jan Hendrik
authors.initials: JH
authors.lastname: Moltz
authors.affiliation: GE Healthcare, Buc, France.
authors.forename: Guillaume
authors.initials: G
authors.lastname: Orieux
authors.affiliation: Moffitt Cancer Center and Research Institute, Tampa, Florida, USA.
authors.forename: Robert J
authors.initials: RJ
authors.lastname: Gillies
authors.affiliation: Moffitt Cancer Center and Research Institute, Tampa, Florida, USA.
authors.forename: Yuhua
authors.initials: Y
authors.lastname: Gu
authors.affiliation: ICON Medical Imaging, Warrington, Pennsylvania, USA.
authors.forename: Ninad
authors.initials: N
authors.lastname: Mantri
authors.affiliation: ICON Medical Imaging, Warrington, Pennsylvania, USA.
authors.forename: Gregory
authors.initials: G
authors.lastname: Goldmacher
authors.affiliation: INTIO, Inc., Broomfield, Colorado, USA.
authors.forename: Luduan
authors.initials: L
authors.lastname: Zhang
authors.affiliation: NYU Langone Medical Center Faculty Practice Radiology, New York, USA.
authors.forename: Emilio
authors.initials: E
authors.lastname: Vega
authors.affiliation: NYU Langone Medical Center Faculty Practice Radiology, New York, USA.
authors.forename: Michael
authors.initials: M
authors.lastname: Bloom
authors.affiliation: Perceptive Informatics, Andhra Pradesh, India.
authors.forename: Rudresh
authors.initials: R
authors.lastname: Jarecha
authors.affiliation: Siemens AG, Healthcare Sector, Computed Tomography, Forchheim, Germany.
authors.forename: Grzegorz
authors.initials: G
authors.lastname: Soza
authors.affiliation: Siemens AG, Healthcare Sector, Computed Tomography, Forchheim, Germany.
authors.forename: Christian
authors.initials: C
authors.lastname: Tietjen
authors.affiliation: Toshiba Corporation, Corporate R&D Center, Kawasaki, Japan.
authors.forename: Tomoyuki
authors.initials: T
authors.lastname: Takeguchi
authors.affiliation: Toshiba Corporation, Toshiba Medical Systems Corporation, Otawara, Japan.
authors.forename: Hitoshi
authors.initials: H
authors.lastname: Yamagata
authors.affiliation: Vital Images, Inc. (a Toshiba Medical Systems Group), Minnesota, USA.
authors.forename: Sam
authors.initials: S
authors.lastname: Peterson
authors.affiliation: Vital Images, Inc. (a Toshiba Medical Systems Group), Minnesota, USA.
authors.forename: Osama
authors.initials: O
authors.lastname: Masoud
authors.affiliation: Buckler Biomedical Associates LLC, Massachusetts, USA.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Buckler
mesh_terms: D000465:Algorithms; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D008168:Lung / Q000000981:diagnostic imaging / Q000473:pathology; D008175:Lung Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology*; D019047:Phantoms, Imaging; D015203:Reproducibility of Results; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging* / Q000473:pathology*; D014057:Tomography, X-Ray Computed / Q000379:methods*; D047368:Tumor Burden
keywords: CT volumetry; QIBA; algorithms; anthropomorphic phantoms; challenge; lung tumor