Projects per year
Abstract
PrecisionRecall analysis abounds in applications of binary classification where true negatives do not add value and hence should not affect assessment of the classifier's performance. Perhaps inspired by the many advantages of receiver operating characteristic (ROC) curves and the area under such curves for accuracybased performance assessment, many researchers have taken to report PrecisionRecall (PR) curves and associated areas as performance metric. We demonstrate in this paper that this practice is fraught with difficulties, mainly because of incoherent scale assumptions  e.g., the area under a PR curve takes the arithmetic mean of precision values whereas the Fβ score applies the harmonic mean. We show how to fix this by plotting PR curves in a different coordinate system, and demonstrate that the new PrecisionRecallGain curves inherit all key advantages of ROC curves. In particular, the area under PrecisionRecallGain curves conveys an expected F1 score on a harmonic scale, and the convex hull of a PrecisionRecallGain curve allows us to calibrate the classifier's scores so as to determine, for each operating point on the convex hull, the interval of β values for which the point optimises Fβ. We demonstrate experimentally that the area under traditional PR curves can easily favour models with lower expected F1 score than others, and so the use of PrecisionRecallGain curves will result in better model selection.
Original language  English 

Title of host publication  Advances in Neural Information Processing Systems 28 
Editors  C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, R. Garnett 
Publisher  Massachusetts Institute of Technology (MIT) Press 
Pages  838846 
Number of pages  9 
Volume  1 
Publication status  Published  7 Dec 2015 
Event  NIPS'15: : Proceedings of the 28th International Conference on Neural Information Processing Systems  Duration: 1 Dec 2015 → … Conference number: 28 
Conference
Conference  NIPS'15: 

Period  1/12/15 → … 
Structured keywords
 Jean Golding
Fingerprint
Dive into the research topics of 'PrecisionRecallGain Curves: PR Analysis Done Right'. Together they form a unique fingerprint.Projects
 1 Finished
Profiles

Professor Peter A Flach
 School of Computer Science  Professor of Artificial Intelligence
 Cabot Institute for the Environment
 Intelligent Systems Laboratory
Person: Academic , Member