title: Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy.
abstract: OBJECTIVE
  Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review.
  
  
  METHODS
  In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters.
  
  
  RESULTS
  It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed.
  
  
  CONCLUSIONS
  A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.
authors.affiliation: Div. Medical Radiation Physics, Department of Radiotherapy, Medical University Vienna/AKH Vienna, Währinger Gürtel 18-20, Vienna, Austria. irina.fotina@meduniwien.ac.at
authors.forename: I
authors.initials: I
authors.lastname: Fotina
authors.forename: C
authors.initials: C
authors.lastname: Lütgendorf-Caucig
authors.forename: M
authors.initials: M
authors.lastname: Stock
authors.forename: R
authors.initials: R
authors.lastname: Pötter
authors.forename: D
authors.initials: D
authors.lastname: Georg
mesh_terms: D005260:Female; D006801:Humans; D008175:Lung Neoplasms / Q000000981:diagnostic imaging* / Q000532:radiotherapy*; D008297:Male; D015588:Observer Variation; D011471:Prostatic Neoplasms / Q000000981:diagnostic imaging* / Q000532:radiotherapy*; D061089:Radiotherapy, Image-Guided / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000379:methods*