Abstract
In this paper we propose methods to detect and repair
concavities in ROC curves by manipulating model predictions. We
introduce two model assembly algorithms. Algorithm SwapOne aims to
improve the Area Under the ROC Curve (AUC) of a probabilistic classifier
by investigating three models from different thresholds of a
probabilistic model, such that one is below the line connecting the
other two, and assembles a hybrid model combining the two better models
and an inversion of the poorer model. Algorithm SwapCurve aims to
improve AUC by identifying part of a probabilistic ROC curve that is
below its convex hull, and inverting the ranking of test instances in
that part of the curve. Experimental results on 24 UCI datasets
demonstrate that the second algorithm gives small but significant
improvements on 10 of these datasets. The novelty of our approach lies
in that, unlike blind ensemble methods, it investigates the performance
of the model in order to decide where performance needs to be improved.
| Translated title of the contribution | Repairing concavities in ROC curves |
|---|---|
| Original language | English |
| Title of host publication | Unknown |
| Publisher | University of Bristol |
| Pages | 38 - 44 |
| Number of pages | 6 |
| Publication status | Published - Aug 2003 |