title: The evaluation of artificial intelligence systems in medicine.
abstract: This paper discusses the underlying issues in the evaluation of computer systems which apply artificial intelligence in medicine. Three different levels of evaluation are described: the subjective evaluation of the research contribution of a developmental prototype, the validation of a system's knowledge and performance, and the evaluation of the clinical efficacy of an operational system. The paper outlines a number of evaluation issues at each level, and discusses how previous artificial intelligence in medicine evaluations fit into this framework.
mesh_terms: D001185:Artificial Intelligence; D003194:Computer-Assisted Instruction; D003657:Decision Making; D003936:Diagnosis, Computer-Assisted; D005069:Evaluation Studies as Topic