title: Unachievable Region in Precision-Recall Space and Its Effect on Empirical Evaluation.
abstract: Precision-recall (PR) curves and the areas under them are widely used to summarize machine learning results, especially for data sets exhibiting class skew. They are often used analogously to ROC curves and the area under ROC curves. It is known that PR curves vary as class skew changes. What was not recognized before this paper is that there is a region of PR space that is completely unachievable, and the size of this region depends only on the skew. This paper precisely characterizes the size of that region and discusses its implications for empirical evaluation methodology in machine learning.
authors.affiliation: University of Wisconsin - Madison, 1300 University Avenue, Madison, WI 53706 USA.
authors.forename: Kendrick
authors.initials: K
authors.lastname: Boyd
authors.affiliation: CRACS INESC-TEC & FCUP, Rua do Campo Alegre, 1021/1055, 4169 - 007 PORTO, Portugal.
authors.forename: Vítor
authors.initials: V
authors.lastname: Santos Costa
authors.affiliation: KU Leuven, Celestijnenlaan 200a, Heverlee 3001, Belgium.
authors.forename: Jesse
authors.initials: J
authors.lastname: Davis
authors.affiliation: University of Wisconsin - Madison, 1300 University Avenue, Madison, WI 53706 USA.
authors.forename: C David
authors.initials: CD
authors.lastname: Page