title: A mammography image set for research purposes using BI-RADS density classification.
abstract: OBJECTIVE
  Breast density categorization consistency is important when performing research, and minimization of interoperator and intraoperator variability is essential. This research aimed to validate a set of mammography images for visual breast density estimation to achieve consistency in future research projects and to determine observer performance.
  
  
  METHODS
  Using the Breast Imaging Reporting and Data System (BI-RADS) as the visual grading scale, 50 mammography images were scored for density grade by 8 observers.
  
  
  RESULTS
  Six of 8 observers achieved near-complete intraobserver agreement (kappa > 0.81). Strong agreement among observers (kappa = 0.61-0.8) was found in 10 of 28 paired observation episodes on the first iteration and 12 of 28 on the second. No observers demonstrated a delta variance above 1. Fleiss' kappa was used to evaluate concordance among all observers on the first and second iterations (first iteration, 0.64; second iteration, 0.56).
  
  
  CONCLUSIONS
  This research illustrates the difficulties of comparing observer visual performance scores because differences can exist when studies are repeated by and among individuals.
  
  
  CONCLUSIONS
  We confirmed that the 50 images were suitable for research purposes. Some variability existed among observers; however, overall density classification agreement was strong. Future research should include repeating this study with digitally acquired images.
authors.forename: Claire Elizabeth
authors.initials: CE
authors.lastname: Mercer
authors.forename: Peter
authors.initials: P
authors.lastname: Hogg
authors.forename: Judith
authors.initials: J
authors.lastname: Kelly
authors.forename: Rita
authors.initials: R
authors.lastname: Borgen
authors.forename: Sara Rachael
authors.initials: SR
authors.lastname: Millington
authors.forename: Beverley
authors.initials: B
authors.lastname: Hilton
authors.forename: David
authors.initials: D
authors.lastname: Enion
authors.forename: Patsy
authors.initials: P
authors.lastname: Whelehan
mesh_terms: D015502:Absorptiometry, Photon / Q000592:standards*; D016208:Databases, Factual / Q000592:standards*; D005260:Female; D017408:Guidelines as Topic; D006801:Humans; D008327:Mammography / Q000379:methods* / Q000592:standards*; D015588:Observer Variation; D011873:Radiology Information Systems / Q000592:standards; D012016:Reference Values; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014481:United States; D064232:Visual Analog Scale