title: Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data.
abstract: OBJECTIVE
  Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile.
  
  
  METHODS
  Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers.
  
  
  RESULTS
  Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail.
  
  
  CONCLUSIONS
  Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.
authors.affiliation: Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984. Electronic address: andrew.buckler@elucidbio.com.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Buckler
authors.affiliation: Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984.
authors.forename: Jovanna
authors.initials: J
authors.lastname: Danagoulian
authors.affiliation: Arbor Analytics LLC, Ann Arbor, Michigan.
authors.forename: Kjell
authors.initials: K
authors.lastname: Johnson
authors.affiliation: National Institute of Standards and Technology, Boulder, Colorado.
authors.forename: Adele
authors.initials: A
authors.lastname: Peskin
authors.affiliation: U.S. Food and Drug Administration, Silver Spring, Maryland.
authors.forename: Marios A
authors.initials: MA
authors.lastname: Gavrielides
authors.affiliation: U.S. Food and Drug Administration, Silver Spring, Maryland.
authors.forename: Nicholas
authors.initials: N
authors.lastname: Petrick
authors.affiliation: Cleveland Clinic, Cleveland, Ohio.
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Obuchowski
authors.affiliation: MEDIAN Technologies, Valbonne, France.
authors.forename: Hubert
authors.initials: H
authors.lastname: Beaumont
authors.affiliation: Department of Radiology, University of Michigan, Ann Arbor, Michigan.
authors.forename: Lubomir
authors.initials: L
authors.lastname: Hadjiiski
authors.affiliation: Perceptive Informatics, Sundew Properties SEZ Pvt Ltd Mindspace, Hyderabad, Andhra Pradesh, India.
authors.forename: Rudresh
authors.initials: R
authors.lastname: Jarecha
authors.affiliation: Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany.
authors.forename: Jan-Martin
authors.initials: JM
authors.lastname: Kuhnigk
authors.affiliation: ICON Medical Imaging, Warrington, Pennsylvania.
authors.forename: Ninad
authors.initials: N
authors.lastname: Mantri
authors.affiliation: Department of Radiology, University of California at Los Angeles, Los Angeles, California.
authors.forename: Michael
authors.initials: M
authors.lastname: McNitt-Gray
authors.affiliation: Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany.
authors.forename: Jan H
authors.initials: JH
authors.lastname: Moltz
authors.affiliation: GE Healthcare, Buc, France.
authors.forename: Gergely
authors.initials: G
authors.lastname: Nyiri
authors.affiliation: Vital Images, Inc., Los Angeles, California.
authors.forename: Sam
authors.initials: S
authors.lastname: Peterson
authors.affiliation: KEOSYS, Saint-Herblain, France.
authors.forename: Pierre
authors.initials: P
authors.lastname: Tervé
authors.affiliation: Siemens AG, Healthcare Sector, Imaging and Therapy Division, Forchheim, Germany.
authors.forename: Christian
authors.initials: C
authors.lastname: Tietjen
authors.affiliation: Mirada Medical Ltd., Oxford Center for Innovation, Oxford, United Kingdom.
authors.forename: Etienne
authors.initials: E
authors.lastname: von Lavante
authors.affiliation: Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984.
authors.forename: Xiaonan
authors.initials: X
authors.lastname: Ma
authors.affiliation: Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984.
authors.forename: Samantha
authors.initials: S
authors.lastname: St Pierre
authors.affiliation: Definiens AG, München, Germany.
authors.forename: Maria
authors.initials: M
authors.lastname: Athelogou
mesh_terms: D000465:Algorithms; D002289:Carcinoma, Non-Small-Cell Lung / Q000000981:diagnostic imaging* / Q000473:pathology*; D005260:Female; D006801:Humans; D016014:Linear Models; D008168:Lung / Q000000981:diagnostic imaging / Q000473:pathology; D008175:Lung Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology*; D015203:Reproducibility of Results; D014057:Tomography, X-Ray Computed; D047368:Tumor Burden
keywords: CT; lung cancer; quantitative imaging; segmentation; volumetry