title: Prospective assessment of an atlas-based intervention combined with real-time software feedback in contouring lymph node levels and organs-at-risk in the head and neck: Quantitative assessment of conformance to expert delineation.
abstract: OBJECTIVE
  A number of studies have previously assessed the role of teaching interventions to improve organ-at-risk (OAR) delineation. We present a preliminary study demonstrating the benefit of a combined atlas and real time software-based feedback intervention to aid in contouring of OARs in the head and neck.
  
  
  METHODS
  The study consisted of a baseline evaluation, a real-time feedback intervention, atlas presentation, and a follow-up evaluation. At baseline evaluation, 8 resident observers contoured 26 OARs on a computed tomography scan without intervention or aid. They then received feedback comparing their contours both statistically and graphically to a set of atlas-based expert contours. Additionally, they received access to an atlas to contour these structures. The resident observers were then asked to contour the same 26 OARs on a separate computed tomography scan with atlas access. In addition, 6 experts (5 radiation oncologists specializing in the head and neck, and 1 neuroradiologist) contoured the 26 OARs on both scans. A simultaneous truth and performance level estimation (STAPLE) composite of the expert contours was used as a gold-standard set for analysis of OAR contouring.
  
  
  RESULTS
  Of the 8 resident observers who initially participated in the study, 7 completed both phases of the study. Dice similarity coefficients were calculated for each user-drawn structure relative to the expert STAPLE composite for each structure. Mean dice similarity coefficients across all structures increased between phase 1 and phase 2 for each resident observer, demonstrating a statistically significant improvement in overall OAR-contouring ability (P < .01). Additionally, intervention improved contouring in 16/26 delineated organs-at-risk across resident observers at a statistically significant level (P ≤ .05) including all otic structures and suprahyoid lymph node levels of the head and neck.
  
  
  CONCLUSIONS
  Our data suggest that a combined atlas and real-time feedback-based educational intervention detectably improves contouring of OARs in the head and neck.
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: Musaddiq
authors.initials: M
authors.lastname: Awan
authors.affiliation: Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Harvard Medical School, Charlestown, Massachusetts; Department of Radiology and Neuroscience, Massachusetts General Hospital, Charlestown, Massachusetts; Department of Radiation Medicine, Oregon Health & Science University, Portland, Oregon.
authors.forename: Jayashree
authors.initials: J
authors.lastname: Kalpathy-Cramer
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: G Brandon
authors.initials: GB
authors.lastname: Gunn
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: Beth M
authors.initials: BM
authors.lastname: Beadle
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: Adam S
authors.initials: AS
authors.lastname: Garden
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: Jack
authors.initials: J
authors.lastname: Phan
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: Emma
authors.initials: E
authors.lastname: Holliday
authors.affiliation: South Texas Veterans Affairs Health Care System, San Antonio, Texas; Department of Radiation Oncology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: William E
authors.initials: WE
authors.lastname: Jones
authors.affiliation: South Texas Veterans Affairs Health Care System, San Antonio, Texas; Department of Radiation Oncology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Maani
authors.affiliation: Department of Radiation Oncology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: Abhilasha
authors.initials: A
authors.lastname: Patel
authors.affiliation: Department of Radiation Oncology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: Jehee
authors.initials: J
authors.lastname: Choi
authors.affiliation: Department of Radiation Oncology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: Virginia
authors.initials: V
authors.lastname: Clyburn
authors.affiliation: Department of Radiology, The University of Texas Health Science Center at San Antonio, San Antonio, Texas.
authors.forename: Bundhit
authors.initials: B
authors.lastname: Tantiwongkosi
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas.
authors.forename: David I
authors.initials: DI
authors.lastname: Rosenthal
authors.affiliation: Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas; Department of Radiation Medicine, Oregon Health & Science University, Portland, Oregon. Electronic address: cdfuller@mdanderson.org.
authors.forename: Clifton D
authors.initials: CD
authors.lastname: Fuller