title: Visualizing the operating range of a classification system.
abstract: The performance of a classification system depends on the context in which it will be used, including the prevalence of the classes and the relative costs of different types of errors. Metrics such as accuracy are limited to the context in which the experiment was originally carried out, and metrics such as sensitivity, specificity, and receiver operating characteristic area--while independent of prevalence--do not provide a clear picture of the performance characteristics of the system over different contexts. Graphing a prevalence-specific metric such as F-measure or the relative cost of errors over a wide range of prevalence allows a visualization of the performance of the system and a comparison of systems in different contexts.
authors.affiliation: Department of Biomedical Informatics, Columbia University Medical Center, 622 West 168th Street, VC5, New York, NY 10027, USA. hripcsak@columbia.edu
authors.forename: George
authors.initials: G
authors.lastname: Hripcsak
mesh_terms: D001296:Audiovisual Aids; D002965:Classification; D003362:Cost-Benefit Analysis; D057225:Data Mining / Q000191:economics; D006801:Humans; D007256:Information Systems / Q000191:economics; D008962:Models, Theoretical; D015995:Prevalence; D012372:ROC Curve; D012680:Sensitivity and Specificity; D013673:Technology Assessment, Biomedical / Q000191:economics / Q000379:methods*