Individualized computer-aided education in mammography based on user modeling: concept and preliminary experiments.
OBJECTIVE
The authors propose the framework for an individualized adaptive computer-aided educational system in mammography that is based on user modeling. The underlying hypothesis is that user models can be developed to capture the individual error making patterns of radiologists-in-training. In this pilot study, the authors test the above hypothesis for the task of breast cancer diagnosis in mammograms.


METHODS
The concept of a user model was formalized as the function that relates image features to the likelihood/extent of the diagnostic error made by a radiologist-in-training and therefore to the level of difficulty that a case will pose to the radiologist-in-training (or "user"). Then, machine learning algorithms were implemented to build such user models. Specifically, the authors explored k-nearest neighbor, artificial neural networks, and multiple regression for the task of building the model using observer data collected from ten Radiology residents at Duke University Medical Center for the problem of breast mass diagnosis in mammograms. For each resident, a user-specific model was constructed that predicts the user's expected level of difficulty for each presented case based on two BI-RADS image features. In the experiments, leave-one-out data handling scheme was applied to assign each case to a low-predicted-difficulty or a high-predicted-difficulty group for each resident based on each of the three user models. To evaluate whether the user model is useful in predicting difficulty, the authors performed statistical tests using the generalized estimating equations approach to determine whether the mean actual error is the same or not between the low-predicted-difficulty group and the high-predicted-difficulty group.


RESULTS
When the results for all observers were pulled together, the actual errors made by residents were statistically significantly higher for cases in the high-predicted-difficulty group than for cases in the low-predicted-difficulty group for all modeling algorithms (p < or = 0.002 for all methods). This indicates that the user models were able to accurately predict difficulty level of the analyzed cases. Furthermore, the authors determined that among the two BI-RADS features that were used in this study, mass margin was the most useful in predicting individual user errors.


CONCLUSIONS
The pilot study shows promise for developing individual user models that can accurately predict the level of difficulty that each case will pose to the radiologist-in-training. These models could allow for constructing adaptive computer-aided educational systems in mammography.
Mazurowski
Maciej A
MA
Department of Radiology, Carl E. Ravin Advanced Imaging Laboratories, Duke University, Durham, North Carolina 27705, USA. maciej.mazurowski@duke.edu
Baker
Jay A
JA
Barnhart
Huiman X
HX
Tourassi
Georgia D
GD
D003194:Computer-Assisted Instruction / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D008327:Mammography; D009657:North Carolina; D010865:Pilot Projects; D011361:Professional Competence; D011871:Radiology / Q000193:education*; D014584:User-Computer Interface
AJR Am J Roentgenol. 1992 Mar;158(3):521-6
1310825
AJR Am J Roentgenol. 1986 Apr;146(4):661-3
3485337
Radiology. 1992 Jul;184(1):39-43
1609100
AJR Am J Roentgenol. 2007 Jan;188(1):236-41
17179372
Radiology. 2005 Jun;235(3):775-90
15914475
N Engl J Med. 1994 Dec 1;331(22):1493-9
7969300
Artif Intell Med. 2006 Sep;38(1):5-24
16183267
N Engl J Med. 2005 Oct 27;353(17):1784-92
16251534
J Am Coll Radiol. 2007 Nov;4(11):752-3
17964496
AJR Am J Roentgenol. 2006 Nov;187(5):W456-68
17056875
Radiology. 2006 May;239(2):385-91
16569780