title: Area under precision-recall curves for weighted and unweighted data.
abstract: Precision-recall curves are highly informative about the performance of binary classifiers, and the area under these curves is a popular scalar performance measure for comparing different classifiers. However, for many applications class labels are not provided with absolute certainty, but with some degree of confidence, often reflected by weights or soft labels assigned to data points. Computing the area under the precision-recall curve requires interpolating between adjacent supporting points, but previous interpolation schemes are not directly applicable to weighted data. Hence, even in cases where weights were available, they had to be neglected for assessing classifiers using precision-recall curves. Here, we propose an interpolation for precision-recall curves that can also be used for weighted data, and we derive conditions for classification scores yielding the maximum and minimum area under the precision-recall curve. We investigate accordances and differences of the proposed interpolation and previous ones, and we demonstrate that taking into account existing weights of test data is important for the comparison of classifiers.
authors.affiliation: Institute for Biosafety in Plant Biotechnology, Julius Kühn-Institut (JKI) - Federal Research Centre for Cultivated Plants, Quedlinburg, Germany.
authors.forename: Jens
authors.initials: J
authors.lastname: Keilwagen
authors.affiliation: Institute of Computer Science, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany; German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, Leipzig, Germany.
authors.forename: Ivo
authors.initials: I
authors.lastname: Grosse
authors.affiliation: Institute of Computer Science, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany.
authors.forename: Jan
authors.initials: J
authors.lastname: Grau
mesh_terms: D000465:Algorithms; D019540:Area Under Curve; D019992:Databases as Topic; D012372:ROC Curve; D013223:Statistics as Topic