title: Modeling false positive error making patterns in radiology trainees for improved mammography education.
abstract: BACKGROUND
  While mammography notably contributes to earlier detection of breast cancer, it has its limitations, including a large number of false positive exams. Improved radiology education could potentially contribute to alleviating this issue. Toward this goal, in this paper we propose an algorithm for modeling of false positive error making among radiology trainees. Identifying troublesome locations for the trainees could focus their training and in turn improve their performance.
  
  
  METHODS
  The algorithm proposed in this paper predicts locations that are likely to result in a false positive error for each trainee based on the previous annotations made by the trainee. The algorithm consists of three steps. First, the suspicious false positive locations are identified in mammograms by Difference of Gaussian filter and suspicious regions are segmented by computer vision-based segmentation algorithms. Second, 133 features are extracted for each suspicious region to describe its distinctive characteristics. Third, a random forest classifier is applied to predict the likelihood of the trainee making a false positive error using the extracted features. The random forest classifier is trained using previous annotations made by the trainee. We evaluated the algorithm using data from a reader study in which 3 experts and 10 trainees interpreted 100 mammographic cases.
  
  
  RESULTS
  The algorithm was able to identify locations where the trainee will commit a false positive error with accuracy higher than an algorithm that selects such locations randomly. Specifically, our algorithm found false positive locations with 40% accuracy when only 1 location was selected for all cases for each trainee and 12% accuracy when 10 locations were selected. The accuracies for randomly identified locations were both 0% for these two scenarios.
  
  
  CONCLUSIONS
  In this first study on the topic, we were able to build computer models that were able to find locations for which a trainee will make a false positive error in images that were not previously seen by the trainee. Presenting the trainees with such locations rather than randomly selected ones may improve their educational outcomes.
authors: Department of Radiology, Duke University School of Medicine, Durham, NC, United States; Computer Science Department, Lamar University, Beaumont, TX, United States. Electronic address: jing.zhang2@duke.edu.
authors: Jing
authors: J
authors: Zhang
authors: Department of Biomedical Engineering, Duke University Pratt School of Engineering, Durham, NC, United States.
authors: James I
authors: JI
authors: Silber
authors: Department of Radiology, Duke University School of Medicine, Durham, NC, United States; Duke Cancer Institute, United States; Duke Medical Physics Program, United States.
authors: Maciej A
authors: MA
authors: Mazurowski
mesh_terms: D000465:Algorithms; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D019295:Computational Biology; D005189:False Positive Reactions; D005260:Female; D006801:Humans; D000069550:Machine Learning; D008327:Mammography / Q000379:methods*; D012372:ROC Curve; D011871:Radiology / Q000193:education*; D014463:Ultrasonography
keywords: Breast cancer; Computer vision; Machine learning; Radiology education