title: Entering the black box of neural networks.
abstract: OBJECTIVE
  Artificial neural networks have proved to be accurate predictive instruments in several medical domains, but have been criticized for failing to specify the information upon which their predictions are based. We used methods of relevance analysis and sensitivity analysis to determine the most important predictor variables for a validated neural network for community-acquired pneumonia.
  
  
  METHODS
  We studied a feed-forward, back-propagation neural network trained to predict pneumonia among patients presenting to an emergency department with fever or respiratory complaints. We used the methods of full retraining, weight elimination, constant substitution, linear substitution, and data permutation to identify a consensus set of important demographic, symptom, sign, and comorbidity predictors that influenced network output for pneumonia. We compared predictors identified by these methods to those identified by a weight propagation analysis based on the matrices of the network, and by logistic regression.
  
  
  RESULTS
  Predictors identified by these methods were clinically plausible, and were concordant with those identified by weight analysis, and by logistic regression using the same data. The methods were highly correlated in network error, and led to variable sets with errors below bootstrap 95% confidence intervals for networks with similar numbers of inputs. Scores for variable relevance tended to be higher with methods that precluded network retraining (weight elimination) or that permuted variable values (data permutation), compared with methods that permitted retraining (full retraining) or that approximated its effects (constant and linear substitution).
  
  
  CONCLUSIONS
  Methods of relevance analysis and sensitivity analysis are useful for identifying important predictor variables used by artificial neural networks.
authors.affiliation: Department of Medicine, University of Illinois, Chicago, IL 60612, USA. pshecker@uic.edu
authors.forename: P S
authors.initials: PS
authors.lastname: Heckerling
authors.forename: B S
authors.initials: BS
authors.lastname: Gerber
authors.forename: T G
authors.initials: TG
authors.lastname: Tape
authors.forename: R S
authors.initials: RS
authors.lastname: Wigton
mesh_terms: D000465:Algorithms; D017714:Community-Acquired Infections / Q000175:diagnosis / Q000503:physiopathology*; D003625:Data Collection; D006801:Humans; D016571:Neural Networks, Computer; D018410:Pneumonia, Bacterial / Q000175:diagnosis / Q000503:physiopathology*; D012680:Sensitivity and Specificity; D014481:United States