title: Interrater reliability of the categorization of late radiographic changes after lung stereotactic body radiation therapy.
abstract: OBJECTIVE
  Radiographic changes after lung stereotactic body radiation therapy (SBRT) have been categorized into 4 groups: modified conventional pattern (A), mass-like fibrosis; (B), scar-like fibrosis (C), and no evidence of increased density (D). The purpose of this study was to assess the interrater reliability of this categorization system in patients with early-stage non-small cell lung cancer (NSCLC).
  
  
  METHODS
  Seventy-seven patients were included in this study, all treated with SBRT for early-stage (T1/2) NSCLC at a single institution, with a minimum follow-up of 6 months. Six experienced clinicians familiar with post-SBRT radiographic changes scored the serial posttreatment CT images independently in a blinded fashion. The proportion of patients categorized as A, B, C, or D at each interval was determined. Krippendorff's alpha (KA), Multirater kappa (M-kappa), and Gwet's AC1 (AC1) scores were used to establish interrater reliability. A leave-one-out analysis was performed to demonstrate the variability among raters. Interrater agreement of the first and last 20 patients scored was calculated to explore whether a training effect existed.
  
  
  RESULTS
  The number of ratings ranged from 450 at 6 months to 84 at 48 months of follow-up. The proportion of patients in each category was as follows: A, 45%; B, 16%; C, 13%; and D, 26%. KA and M-kappa ranged from 0.17 to 0.34. AC1 measure range was 0.22 to 0.48. KA increased from 0.24 to 0.36 at 12 months with training. The percent agreement for pattern A peaked at 12 month with a 54% chance of having >50% raters in agreement and decreased over time, whereas that for patterns B and C increased over time to a maximum of 20% and 22%, respectively.
  
  
  CONCLUSIONS
  This post-SBRT radiographic change categorization system has modest interrater agreement, and there is a suggestion of a training effect. Patterns of fibrosis evolve after SBRT and alternative categorization systems should be evaluated.
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Salman
authors.initials: S
authors.lastname: Faruqi
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada. Electronic address: meredith.giuliani@rmp.uhn.on.ca.
authors.forename: Meredith E
authors.initials: ME
authors.lastname: Giuliani
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Hamid
authors.initials: H
authors.lastname: Raziee
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Mei Ling
authors.initials: ML
authors.lastname: Yap
authors.affiliation: Department of Radiology, University Health Network, Toronto, Ontario, Canada.
authors.forename: Heidi
authors.initials: H
authors.lastname: Roberts
authors.affiliation: Department of Biostatistics, Princess Margaret Cancer Centre, Toronto, Ontario, Canada.
authors.forename: Lisa W
authors.initials: LW
authors.lastname: Le
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Anthony
authors.initials: A
authors.lastname: Brade
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: John
authors.initials: J
authors.lastname: Cho
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Alexander
authors.initials: A
authors.lastname: Sun
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Andrea
authors.initials: A
authors.lastname: Bezjak
authors.affiliation: Department of Radiation Oncology, Princess Margaret Cancer Centre, Toronto, ON, Canada.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Hope
mesh_terms: D000368:Aged; D000369:Aged, 80 and over; D002289:Carcinoma, Non-Small-Cell Lung / Q000473:pathology / Q000601:surgery*; D005260:Female; D006801:Humans; D008168:Lung / Q000000981:diagnostic imaging* / Q000528:radiation effects*; D008175:Lung Neoplasms / Q000473:pathology / Q000601:surgery*; D008297:Male; D008875:Middle Aged; D009364:Neoplasm Recurrence, Local / Q000000981:diagnostic imaging; D009367:Neoplasm Staging; D015588:Observer Variation; D017564:Radiation Pneumonitis / Q000000981:diagnostic imaging* / Q000453:epidemiology; D016634:Radiosurgery / Q000009:adverse effects* / Q000379:methods; D015203:Reproducibility of Results; D014057:Tomography, X-Ray Computed; D014741:Video Recording