title: Variability in Observer Performance Between Faculty Members and Residents Using Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound, Fifth Edition (2013).
abstract: BACKGROUND
  Ultrasonography (US) is a useful tool for breast imaging, yet is highly operator-dependent.
  
  
  OBJECTIVE
  We evaluated inter-observer variability and performance discrepancies between faculty members and radiology residents when describing breast lesions, by the fifth edition of breast imaging reporting and data system (BI-RADS)-US lexicon, and then attempted to identify whether inter-observer variability could be improved after one education session.
  
  
  METHODS
  In total, 50 malignant lesions and 70 benign lesions were considered in our retrospective study. Two faculty members, two senior residents, and two junior residents separately assessed the US images. After the first assessment, the readers received one education session, and then reassessed the images in a random order. Inter-observer variability was measured using the kappa coefficient (κ). Performance discrepancy was evaluated by receiver operating characteristic (ROC) curves.
  
  
  RESULTS
  For the faculty members, fair-to-good agreement was obtained in all descriptors and final assessment, while for residents, poor-to-moderate agreement was obtained. The areas under the ROC curves were 0.78 for the faculty members, 0.59 for the senior residents, and 0.52 for the junior residents, respectively. Diagnostic performance was significantly higher in the faculty members than the senior and junior residents (P = 0.0001 and < 0.0001, respectively). After one education session, the agreement in the final assessment was one level higher in the faculty members and senior residents, yet in the senior residents, the degree of agreement was still only fair. Moreover, in the junior residents, there was no improvement.
  
  
  CONCLUSIONS
  Investigative assessment of breast US by residents is inadvisable. We recommend continued professional resident training to improve the degree of agreement and performance.
authors.affiliation: Department of Radiology, Daejeon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Daejeon, Republic of Korea.
authors.forename: Youn Joo
authors.initials: YJ
authors.lastname: Lee
authors.affiliation: Department of Radiology, Eulji University Hospital, Daejeon, Republic of Korea.
authors.forename: So Young
authors.initials: SY
authors.lastname: Choi
authors.affiliation: Department of Radiology, Eulji University Hospital, Daejeon, Republic of Korea.
authors.forename: Kyu Sun
authors.initials: KS
authors.lastname: Kim
authors.affiliation: Department of Radiology, Daejeon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Daejeon, Republic of Korea.
authors.forename: Po Song
authors.initials: PS
authors.lastname: Yang
keywords: Breast; Faculty; Neoplasms; Residency; Ultrasonography