title: Performance of four computer-based diagnostic systems.
abstract: BACKGROUND
  Computer-based diagnostic systems are available commercially, but there has been limited evaluation of their performance. We assessed the diagnostic capabilities of four internal medicine diagnostic systems: Dxplain, Iliad, Meditel, and QMR.
  
  
  METHODS
  Ten expert clinicians created a set of 105 diagnostically challenging clinical case summaries involving actual patients. Clinical data were entered into each program with the vocabulary provided by the program's developer. Each of the systems produced a ranked list of possible diagnoses for each patient, as did the group of experts. We calculated scores on several performance measures for each computer program.
  
  
  RESULTS
  No single computer program scored better than the others on all performance measures. Among all cases and all programs, the proportion of correct diagnoses ranged from 0.52 to 0.71, and the mean proportion of relevant diagnoses ranged from 0.19 to 0.37. On average, less than half the diagnoses on the experts' original list of reasonable diagnoses were suggested by any of the programs. However, each program suggested an average of approximately two additional diagnoses per case that the experts found relevant but had not originally considered.
  
  
  CONCLUSIONS
  The results provide a profile of the strengths and limitations of these computer programs. The programs should be used by physicians who can identify and use the relevant information and ignore the irrelevant information that can be produced.
authors.affiliation: University of Alabama at Birmingham.
authors.forename: E S
authors.initials: ES
authors.lastname: Berner
authors.forename: G D
authors.initials: GD
authors.lastname: Webster
authors.forename: A A
authors.initials: AA
authors.lastname: Shugerman
authors.forename: J R
authors.initials: JR
authors.lastname: Jackson
authors.forename: J
authors.initials: J
authors.lastname: Algina
authors.forename: A L
authors.initials: AL
authors.lastname: Baker
authors.forename: E V
authors.initials: EV
authors.lastname: Ball
authors.forename: C G
authors.initials: CG
authors.lastname: Cobbs
authors.forename: V W
authors.initials: VW
authors.lastname: Dennis
authors.forename: E P
authors.initials: EP
authors.lastname: Frenkel
mesh_terms: D000704:Analysis of Variance; D003936:Diagnosis, Computer-Assisted / Q000592:standards*; D005069:Evaluation Studies as Topic; D006801:Humans; D007388:Internal Medicine / Q000592:standards*; D012984:Software / Q000592:standards*