Comparison of the validity and reliability of two image classification systems for the assessment of mammogram quality.
OBJECTIVE
To compare the reliability and validity of two classification systems used to evaluate the quality of mammograms: PGMI ('perfect', 'good', 'moderate' and 'inadequate') and EAR ('excellent', 'acceptable' and 'repeat').


METHODS
New South Wales (Australia) population-based mammography screening programme (BreastScreen NSW).


METHODS
Thirty sets of mammograms were rated by 21 radiographers and an expert panel. PGMI and EAR criteria were used to assign ratings to the medio-lateral oblique (MLO) and cranio-caudal (CC) views for each set of films. Inter-observer reliability and criterion validity (compared with expert panel ratings) were assessed using mean weighted observed agreement and kappa statistics.


RESULTS
Reliability: Kappa values for both classification systems were low (0.01-0.17). PGMI produced significantly higher values than EAR. Agreement between raters was higher using PGMI than EAR for the MLO view (77% versus 74%, P < 0.05), but was similar for the CC view. Dichotomized ratings ('acceptable' or 'needs repeating') did not improve reliability estimates.


RESULTS
Kappa values between raters and the reference standard were low for both classification systems (0.05-0.15). Agreement between raters and the reference standard was higher using PGMI than EAR for the MLO view (74% versus 63%), but was similar for the CC view. Dichotomized ratings of the MLO view showed slightly higher observer agreement.


CONCLUSIONS
Both PGMI and EAR have poor reliability and validity in evaluating mammogram quality. EAR is not a suitable alternative to PGMI, which must be improved if it is to be useful.
Moreira
Conrad
C
NSW Public Health Officer Training Program, NSW Department of Health, Australia.
Svoboda
Kate
K
Poulos
Ann
A
Taylor
Richard
R
Page
Andrew
A
Rickard
Mary
M
D001943:Breast Neoplasms / Q000175:diagnosis / Q000517:prevention & control; D005260:Female; D006801:Humans; D008327:Mammography / Q000145:classification / Q000592:standards*; D008403:Mass Screening / Q000145:classification / Q000592:standards*; D009517:New South Wales; D011785:Quality Assurance, Health Care; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000592:standards; D015203:Reproducibility of Results