ROC curves in cost space

José Hernández-Orallo, Peter Flach, César Ferri*

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

20 Citations (Scopus)


ROC curves and cost curves are two popular ways of visualising classifier performance, finding appropriate thresholds according to the operating condition, and deriving useful aggregated measures such as the area under the ROC curve (AUC) or the area under the optimal cost curve. In this paper we present new findings and connections between ROC space and cost space. In particular, we show that ROC curves can be transferred to cost space by means of a very natural threshold choice method, which sets the decision threshold such that the proportion of positive predictions equals the operating condition. We call these new curves rate-driven curves, and we demonstrate that the expected loss as measured by the area under these curves is linearly related to AUC. We show that the rate-driven curves are the genuine equivalent of ROC curves in cost space, establishing a point-point rather than a point-line correspondence. Furthermore, a decomposition of the rate-driven curves is introduced which separates the loss due to the threshold choice method from the ranking loss (Kendall tau distance). We also derive the corresponding curve to the ROC convex hull in cost space; this curve is different from the lower envelope of the cost lines, as the latter assumes only optimal thresholds are chosen.

Original languageEnglish
Pages (from-to)71-91
Number of pages21
JournalMachine Learning
Issue number1
Publication statusPublished - 1 Oct 2013

Structured keywords

  • Jean Golding


  • Area Under the ROC Curve (AUC)
  • Cost curves
  • Cost-sensitive evaluation
  • Kendall tau distance
  • Operating condition
  • Ranking performance
  • ROC curves


Dive into the research topics of 'ROC curves in cost space'. Together they form a unique fingerprint.

Cite this