title: Categorizing segmentation quality using a quantitative quality assurance algorithm.
abstract: BACKGROUND
  Obtaining high levels of contouring consistency is a major limiting step in optimizing the radiotherapeutic ratio. We describe a novel quantitative methodology for the quality assurance (QA) of contour compliance referenced against a community set of contouring experts.
  
  
  METHODS
  Two clinical tumour site scenarios (10 lung cases and one prostate case) were used with QA algorithm. For each case, multiple physicians (lung: n = 6, prostate: n = 25) segmented various target/organ at risk (OAR) structures to define a set of community reference contours. For each set of community contours, a consensus contour (Simultaneous Truth and Performance Level Estimation (STAPLE)) was created. Differences between each individual community contour versus the group consensus contour were quantified by consensus-based contouring penalty metric (PM) scores. New observers segmented these same cases to calculate individual PM scores (for each unique target/OAR) for each new observer-STAPLE pair for comparison against the community and consensus contours.
  
  
  RESULTS
  Four physicians contoured the 10 lung cases for a total of 72 contours for quality assurance evaluation against the previously derived community consensus contours. A total of 16 outlier contours were identified by the QA system of which 11 outliers were due to over-contouring discrepancies, three were due to over-/under-contouring discrepancies, and two were due to missing/incorrect nodal contours. In the prostate scenario involving six physicians, the QA system detected a missing penile bulb contour, systematic inner-bladder contouring, and under-contouring of the upper/anterior rectum.
  
  
  CONCLUSIONS
  A practical methodology for QA has been demonstrated with future clinical trial credentialing, medical education and auto-contouring assessment applications.
authors.affiliation: Department of Radiation Oncology, University of Western Ontario and Lawson Health Research Institute, Canada. george.rodrigues@lhsc.on.ca
authors.forename: George
authors.initials: G
authors.lastname: Rodrigues
authors.forename: Alexander
authors.initials: A
authors.lastname: Louie
authors.forename: Gregory
authors.initials: G
authors.lastname: Videtic
authors.forename: Lara
authors.initials: L
authors.lastname: Best
authors.forename: Nikhilesh
authors.initials: N
authors.lastname: Patil
authors.forename: Abhirami
authors.initials: A
authors.lastname: Hallock
authors.forename: Stewart
authors.initials: S
authors.lastname: Gaede
authors.forename: Jeff
authors.initials: J
authors.lastname: Kempe
authors.forename: Jerry
authors.initials: J
authors.lastname: Battista
authors.forename: Patricia
authors.initials: P
authors.lastname: de Haan
authors.forename: Glenn
authors.initials: G
authors.lastname: Bauman
mesh_terms: D002170:Canada; D006801:Humans; D009369:Neoplasms / Q000000981:diagnostic imaging* / Q000532:radiotherapy*; D011785:Quality Assurance, Health Care / Q000592:standards*; D011879:Radiotherapy Dosage; D011880:Radiotherapy Planning, Computer-Assisted / Q000379:methods*; D020266:Radiotherapy, Conformal / Q000592:standards*; D061089:Radiotherapy, Image-Guided / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000592:standards*