title: Reproducibility--an important factor determining the quality of computer-aided detection (CAD) systems.
abstract: OBJECTIVE
  To test the reproducibility of markings on mammography films set by a commercial computer aided detection (CAD) system.
  
  
  METHODS
  One hundred unilateral mammography examinations (each in CC and MLO) of 100 patients with mammographically detected suspicious foci, which were histopathologically proven to be malignant, were scanned three times with the CAD system, retrospectively. Every fifth patient of the institutional tumor case sampler was enrolled in the study. Only cases with one visible lesion were included in the study. Reproducibility and sensitivity (in both the strict and the broader sense) were determined. Strict sensitivity means the correct set of markers in both images, whereas broader sensitivity means the correct set in at least one of the images. Sixteen of 100 malignancies were indicated by focal suspicious microcalcification clusters, 53 tumors by masses and 31 cases by both signs of breast cancer. The CAD evaluation was divided into only two different markers: one for microcalcifications and one for masses. Thus, 47 (16+31) tumor-induced microcalcifications and 84 (53+31) malignancy-related masses were checked using the CAD system.
  
  
  RESULTS
  Eighteen of 100 unilateral mammography examinations revealed identical patterns in all three scans (18% reproducibility). Eleven of 47 suspicious focal microcalcification clusters and 43/84 masses were correctly marked on both mammographic views in all three CAD scans (strict and broader sensitivity, 23.4 and 51.1%, respectively). Six of 47 microcalcification clusters and 8/84 masses were totally missed in all images by the system (false negative rate, 12.8 and 9.6%, respectively).
  
  
  CONCLUSIONS
  Reproducibility is essential for CAD systems. Currently, reproducibility of the used CAD system appears to be insufficient for clinical routine. Improvement of the system characteristics would make such systems valuable as a 'second reader' in clinical examination.
authors.affiliation: Institute of Diagnostic and Interventional Radiology, Friedrich-Schiller-University Jena, Bachstrasse 18, 07740 Jena, Germany. ansgarmalich@gmx.de
authors.forename: A
authors.initials: A
authors.lastname: Malich
authors.forename: T
authors.initials: T
authors.lastname: Azhari
authors.forename: T
authors.initials: T
authors.lastname: Böhm
authors.forename: M
authors.initials: M
authors.lastname: Fleck
authors.forename: W A
authors.initials: WA
authors.lastname: Kaiser
mesh_terms: D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D005188:False Negative Reactions; D005260:Female; D006801:Humans; D008327:Mammography / Q000295:instrumentation / Q000379:methods / Q000592:standards*; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000295:instrumentation / Q000379:methods / Q000592:standards*; D015203:Reproducibility of Results; D012189:Retrospective Studies; D012680:Sensitivity and Specificity