Rate-oriented point-wise confidence bounds for ROC curves

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

3 Citations (Scopus)

Abstract

Common approaches to generating confidence bounds around ROC curves have several shortcomings. We resolve these weaknesses with a new 'rate-oriented' approach. We generate confidence bounds composed of a series of confidence intervals for a consensus curve, each at a particular predicted positive rate (PPR), with the aim that each confidence interval contains new samples of this consensus curve with probability 95%. We propose two approaches; a parametric and a bootstrapping approach, which we base on a derivation from first principles. Our method is particularly appropriate with models used for a common type of task that we call rate-constrained, where a certain proportion of examples needs to be classified as positive by the model, such that the operating point will be set at a particular PPR value. © 2014 Springer-Verlag.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
EditorsToon Calders, Floriana Esposito, Eyke Hullermeier, Rosa Meo
PublisherSpringer Berlin Heidelberg
Pages404-421
Number of pages18
Volume8725 LNAI
EditionPART 2
ISBN (Print)9783662448502
DOIs
Publication statusPublished - 1 Jan 2014
EventEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014 - Nancy, France
Duration: 15 Sep 201419 Sep 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 2
Volume8725 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Conference

ConferenceEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014
CountryFrance
CityNancy
Period15/09/1419/09/14

Structured keywords

  • Jean Golding

Keywords

  • Confidence bounds
  • rate-averaging
  • ROC curves
  • rate-constrained

Fingerprint

Dive into the research topics of 'Rate-oriented point-wise confidence bounds for ROC curves'. Together they form a unique fingerprint.

Cite this