title: Comparison of the validity and reliability of two image classification systems for the assessment of mammogram quality.
abstract: OBJECTIVE
  To compare the reliability and validity of two classification systems used to evaluate the quality of mammograms: PGMI ('perfect', 'good', 'moderate' and 'inadequate') and EAR ('excellent', 'acceptable' and 'repeat').
  
  
  METHODS
  New South Wales (Australia) population-based mammography screening programme (BreastScreen NSW).
  
  
  METHODS
  Thirty sets of mammograms were rated by 21 radiographers and an expert panel. PGMI and EAR criteria were used to assign ratings to the medio-lateral oblique (MLO) and cranio-caudal (CC) views for each set of films. Inter-observer reliability and criterion validity (compared with expert panel ratings) were assessed using mean weighted observed agreement and kappa statistics.
  
  
  RESULTS
  Reliability: Kappa values for both classification systems were low (0.01-0.17). PGMI produced significantly higher values than EAR. Agreement between raters was higher using PGMI than EAR for the MLO view (77% versus 74%, P < 0.05), but was similar for the CC view. Dichotomized ratings ('acceptable' or 'needs repeating') did not improve reliability estimates.
  
  
  RESULTS
  Kappa values between raters and the reference standard were low for both classification systems (0.05-0.15). Agreement between raters and the reference standard was higher using PGMI than EAR for the MLO view (74% versus 63%), but was similar for the CC view. Dichotomized ratings of the MLO view showed slightly higher observer agreement.
  
  
  CONCLUSIONS
  Both PGMI and EAR have poor reliability and validity in evaluating mammogram quality. EAR is not a suitable alternative to PGMI, which must be improved if it is to be useful.
authors.affiliation: NSW Public Health Officer Training Program, NSW Department of Health, Australia.
authors.forename: Conrad
authors.initials: C
authors.lastname: Moreira
authors.forename: Kate
authors.initials: K
authors.lastname: Svoboda
authors.forename: Ann
authors.initials: A
authors.lastname: Poulos
authors.forename: Richard
authors.initials: R
authors.lastname: Taylor
authors.forename: Andrew
authors.initials: A
authors.lastname: Page
authors.forename: Mary
authors.initials: M
authors.lastname: Rickard
mesh_terms: D001943:Breast Neoplasms / Q000175:diagnosis / Q000517:prevention & control; D005260:Female; D006801:Humans; D008327:Mammography / Q000145:classification / Q000592:standards*; D008403:Mass Screening / Q000145:classification / Q000592:standards*; D009517:New South Wales; D011785:Quality Assurance, Health Care; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000592:standards; D015203:Reproducibility of Results