title: Quantitative assessment of target delineation variability for thymic cancers: Agreement evaluation of a prospective segmentation challenge.
abstract: OBJECTIVE
  We sought to quantitatively determine the inter-observer variability of expert radiotherapy target-volume delineation for thymic cancers, as part of a larger effort to develop an expert-consensus contouring atlas.
  
  
  METHODS
  A pilot dataset was created consisting of a standardized case presentation with pre- and post-operative DICOM CT image sets from a single patient with Masaoka-Koga Stage III thymoma. Expert thoracic radiation oncologists delineated tumor targets on the pre- and post-operative scans as they would for a definitive and adjuvant case, respectively. Respondents completed a survey including recommended dose prescription and target volume margins for definitive and post-operative scenarios. Inter-observer variability was analyzed quantitatively with Warfield's simultaneous truth, performance level estimation (STAPLE) algorithm and Dice similarity coefficient (DSC).
  
  
  RESULTS
  Seven users completed contouring for definitive and adjuvant cases; of these, 5 completed online surveys. Segmentation performance was assessed, with high mean±SD STAPLE-estimated segmentation sensitivity for definitive case GTV and CTV at 0.77 and 0.80, respectively, and post-operative CTV sensitivity of 0.55; all volumes had specificity of ≥0.99. Inter-observer agreement was markedly higher for the definitive target volumes, with mean±SD DSC of 0.88±0.03 and 0.89±0.04 for GTV and CTV respectively, compared to post-op CTV DSC of 0.69±0.06 (Kruskal-Wallis p<0.01.
  
  
  CONCLUSIONS
  Expert agreement for definitive case volumes was exceptionally high, though significantly lower agreement was noted post-operatively. Technique and dose prescription between experts was substantively consistent, and these preliminary results will be utilized to create an expert-consensus contouring atlas to aid the non-expert radiation oncologist in the planning of these challenging, rare tumors.
authors: Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors: Emma
authors: E
authors: Holliday
authors: Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors: Clifton D
authors: CD
authors: Fuller
authors: Department of Radiology, Massachusetts General Hospital, Boston, MA.
authors: Jayashree
authors: J
authors: Kalpathy-Cramer
authors: Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors: Daniel
authors: D
authors: Gomez
authors: Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY.
authors: Andreas
authors: A
authors: Rimner
authors: Department of Radiation Oncology, The University of Texas Health Science Center San Antonio, San Antonio, TX.
authors: Ying
authors: Y
authors: Li
authors: Department of Radiation Oncology, University Medical Center, Amsterdam, The Netherlands.
authors: Suresh
authors: S
authors: Senan
authors: Department of Therapeutic Radiology, Yale Cancer Center, New Haven, CT.
authors: Lynn D
authors: LD
authors: Wilson
authors: Department of Radiation Oncology, Kaiser Permanente Southern California, Los Angeles, CA.
authors: Jehee
authors: J
authors: Choi
authors: Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors: Ritsuko
authors: R
authors: Komaki
authors: Department of Radiation Medicine, Oregon Health and Science University Knight Cancer Center, Portland, OR.
authors: Charles R
authors: CR
authors: Thomas
keywords: Radiotherapy; consensus guidelines; target delineation; thymic carcinoma; thymoma; treatment planning