title: Toward semi-automated assessment of target volume delineation in radiotherapy trials: the SCOPE 1 pretrial test case.
abstract: OBJECTIVE
  To evaluate different conformity indices (CIs) for use in the analysis of outlining consistency within the pretrial quality assurance (Radiotherapy Trials Quality Assurance [RTTQA]) program of a multicenter chemoradiation trial of esophageal cancer and to make recommendations for their use in future trials.
  
  
  METHODS
  The National Cancer Research Institute SCOPE 1 trial is an ongoing Cancer Research UK-funded phase II/III randomized controlled trial of chemoradiation with capecitabine and cisplatin with or without cetuximab for esophageal cancer. The pretrial RTTQA program included a detailed radiotherapy protocol, an educational package, and a single mid-esophageal tumor test case that were sent to each investigator to outline. Investigator gross tumor volumes (GTVs) were received from 50 investigators in 34 UK centers, and CERR (Computational Environment for Radiotherapy Research) was used to perform an assessment of each investigator GTV against a predefined gold-standard GTV using different CIs. A new metric, the local conformity index (l-CI), that can localize areas of maximal discordance was developed.
  
  
  RESULTS
  The median Jaccard conformity index (JCI) was 0.69 (interquartile range, 0.62-0.70), with 14 of 50 investigators (28%) achieving a JCI of 0.7 or greater. The median geographical miss index was 0.09 (interquartile range, 0.06-0.16), and the mean discordance index was 0.27 (95% confidence interval, 0.25-0.30). The l-CI was highest in the middle section of the volume, where the tumor was bulky and more easily definable, and identified 4 slices where fewer than 20% of investigators achieved an l-CI of 0.7 or greater.
  
  
  CONCLUSIONS
  The available CIs analyze different aspects of a gold standard-observer variation, with JCI being the most useful as a single metric. Additional information is provided by the l-CI and can focus the efforts of the RTTQA team in these areas, possibly leading to semi-automated outlining assessment.
authors.affiliation: Department of Clinical Oncology, Velindre Cancer Centre, Cardiff, Wales. Sarah.Gwynne2@wales.nhs.uk
authors.forename: Sarah
authors.initials: S
authors.lastname: Gwynne
authors.forename: Emiliano
authors.initials: E
authors.lastname: Spezi
authors.forename: Lucy
authors.initials: L
authors.lastname: Wills
authors.forename: Lisette
authors.initials: L
authors.lastname: Nixon
authors.forename: Chris
authors.initials: C
authors.lastname: Hurt
authors.forename: George
authors.initials: G
authors.lastname: Joseph
authors.forename: Mererid
authors.initials: M
authors.lastname: Evans
authors.forename: Gareth
authors.initials: G
authors.lastname: Griffiths
authors.forename: Tom
authors.initials: T
authors.lastname: Crosby
authors.forename: John
authors.initials: J
authors.lastname: Staffurth
mesh_terms: D000911:Antibodies, Monoclonal / Q000008:administration & dosage; D061067:Antibodies, Monoclonal, Humanized; D000971:Antineoplastic Combined Chemotherapy Protocols / Q000627:therapeutic use; D000069287:Capecitabine; D000068818:Cetuximab; D059248:Chemoradiotherapy / Q000379:methods; D002945:Cisplatin / Q000008:administration & dosage; D003841:Deoxycytidine / Q000008:administration & dosage / Q000031:analogs & derivatives; D004938:Esophageal Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology / Q000532:radiotherapy*; D005472:Fluorouracil / Q000008:administration & dosage / Q000031:analogs & derivatives; D006801:Humans; D015588:Observer Variation; D011785:Quality Assurance, Health Care; D011859:Radiography; D047368:Tumor Burden
chemical_list: D000911:Antibodies, Monoclonal; D061067:Antibodies, Monoclonal, Humanized; D003841:Deoxycytidine; D000069287:Capecitabine; D000068818:Cetuximab; D002945:Cisplatin; D005472:Fluorouracil