title: Inter-rater agreement in the characterization of cystic renal lesions on contrast-enhanced MRI.
abstract: OBJECTIVE
  To evaluate inter-observer agreement of MRI features and classification of cystic renal masses among radiologist and radiology trainees.
  
  
  METHODS
  Four readers (two radiologists and two radiology trainees) retrospectively reviewed 100 cystic renal lesions on gadolinium enhanced MRI and assigned each a Bosniak classification (1, 2, 2F, 3, and 4). Lesions were also assessed on their individual features including size, presence of nodules, septations, and enhancement. Readers ranked their level of confidence regarding Bosniak classifications. Inter-observer variability of lesion classification and features was evaluated between raters at both radiologist and radiology trainee levels as well as the level of agreement of all four readers using weighted Kappa and intraclass correlation coefficient (ICC).
  
  
  RESULTS
  One hundred cystic renal lesions were evaluated. There was moderate and substantial classification agreement between trainees and radiologists (ICC 0.59 and 0.63, respectively). There was substantial classification agreement among all four readers (0.66) with the lowest level of agreement for Bosniak 2F lesions (ICC 0.14). There was moderate-substantial agreement for the presence of nodular component, septations, and enhancement. Staff demonstrated highest agreement when assessing for nodular components (0.73). Agreement for the presence of enhancement was lowest (0.37 and 0.42 for radiologists and trainees, respectively). Reported confidence was higher among radiologists compared with trainees.
  
  
  CONCLUSIONS
  There is substantial overall inter-observer agreement in the MRI classification of cystic renal lesions. Confidence increases as rater experience increases.
authors.affiliation: The Ottawa Hospital, General Campus, 501 Smyth Road, Ottawa, ON, K1S 8L6, Canada, nhsppl@mta.ca.
authors.forename: Nicholas
authors.initials: N
authors.lastname: Seppala
authors.forename: Ania
authors.initials: A
authors.lastname: Kielar
authors.forename: Dominique
authors.initials: D
authors.lastname: Dabreo
authors.forename: Shauna
authors.initials: S
authors.lastname: Duigenan
mesh_terms: D002292:Carcinoma, Renal Cell / Q000175:diagnosis*; D003287:Contrast Media; D005682:Gadolinium; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007668:Kidney / Q000473:pathology; D052177:Kidney Diseases, Cystic / Q000175:diagnosis*; D007680:Kidney Neoplasms / Q000175:diagnosis*; D008279:Magnetic Resonance Imaging / Q000379:methods / Q000706:statistics & numerical data*; D015588:Observer Variation; D012189:Retrospective Studies; D012680:Sensitivity and Specificity
chemical_list: D003287:Contrast Media; D005682:Gadolinium