Decision threshold adjustment in class prediction.
Standard classification algorithms are generally designed to maximize the number of correct predictions (concordance). The criterion of maximizing the concordance may not be appropriate in certain applications. In practice, some applications may emphasize high sensitivity (e.g., clinical diagnostic tests) and others may emphasize high specificity (e.g., epidemiology screening studies). This paper considers effects of the decision threshold on sensitivity, specificity, and concordance for four classification methods: logistic regression, classification tree, Fisher's linear discriminant analysis, and a weighted k-nearest neighbor. We investigated the use of decision threshold adjustment to improve performance of either sensitivity or specificity of a classifier under specific conditions. We conducted a Monte Carlo simulation showing that as the decision threshold increases, the sensitivity decreases and the specificity increases; but, the concordance values in an interval around the maximum concordance are similar. For specified sensitivity and specificity levels, an optimal decision threshold might be determined in an interval around the maximum concordance that meets the specified requirement. Three example data sets were analyzed for illustrations.
Chen
J J
JJ
Division of Biometry and Risk Assessment, National Center for Toxicological Research, Food and Drug Administration, Jefferson, Arkansas 72079, USA. jchen@nctr.fda.gov
Tsai
C-A
CA
Moon
H
H
Ahn
H
H
Young
J J
JJ
Chen
C-H
CH
D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D002965:Classification; D003110:Colonic Neoplasms / Q000235:genetics; D003198:Computer Simulation; D016208:Databases, Factual; D003661:Decision Support Techniques; D003663:Decision Trees; D016002:Discriminant Analysis; D020869:Gene Expression Profiling; D006801:Humans; D008113:Liver Neoplasms / Q000139:chemically induced; D016015:Logistic Models; D009010:Monte Carlo Method; D011960:Receptors, Estrogen / Q000378:metabolism; D013329:Structure-Activity Relationship
D011960:Receptors, Estrogen