The effect of data sampling on the performance evaluation of artificial neural networks in medical diagnosis.
OBJECTIVE
To study the effect of data sampling on the predictive assessment of artificial neural networks (ANNs) for medical diagnostic tasks.


METHODS
Three statistical techniques were used to evaluate the diagnostic performances of ANNs: 1) cross validation, 2) round robin, and 3) bootstrap. These techniques are different sampling plans designed to reduce the small-sample estimation bias and variance contributions. The study was based on two networks, one developed for the diagnosis of pulmonary embolism (1,064 cases) and the other developed for the diagnosis of breast cancer (206 cases).


RESULTS
The three sampling techniques produced different performance estimates for both networks. The estimates varied substantially depending on the training sample size and the training-stopping criterion.


CONCLUSIONS
The predictive assessment of ANNs in medical diagnosis can vary substantially based on the complexity of the problem, the data sampling technique, and the number of cases available.
Tourassi
G D
GD
Department of Radiology, Duke University Medical Center, Durham, North Carolina, USA.
Floyd
C E
CE
D001943:Breast Neoplasms / Q000175:diagnosis; D003625:Data Collection; D003661:Decision Support Techniques; D003936:Diagnosis, Computer-Assisted / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D008327:Mammography / Q000706:statistics & numerical data; D015233:Models, Statistical; D016571:Neural Networks, Computer; D011655:Pulmonary Embolism / Q000175:diagnosis; D012372:ROC Curve; D015203:Reproducibility of Results