Background Check: A General Technique to Build More Reliable and Versatile Classifiers

Miquel Perello-Nieto, Telmo M Silva Filho, Meelis Kull, Peter Flach

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

4 Citations (Scopus)
446 Downloads (Pure)

Abstract

We introduce a powerful technique to make classifiers more reliable and versatile. Background Check equips classifiers with the ability to assess the difference of unlabelled test data from the training data. In particular, Background Check gives classifiers the capability to (i) perform cautious classification with a reject option; (ii) identify outliers; and (iii) better assess the confidence in their predictions. We derive the method from first principles and consider four particular relationships between background and foreground distributions. One of these assumes an affine relationship with two parameters, and we show how this bivariate parameter space naturally interpolates between the above capabilities.We demonstrate the versatility of the approach by comparing it experimentally with published special-purpose solutions for outlier detection and confident classification on 41 benchmark datasets. Results show that Background Check can match and in many cases surpass the performances of specialised approaches.

Original languageEnglish
Title of host publication2016 IEEE 16th International Conference on Data Mining (ICDM 2016)
Subtitle of host publicationProceedings of a meeting held 12-15 December 2016, Barcelona, Spain
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1143-1148
Number of pages6
ISBN (Electronic)9781509054725
ISBN (Print)9781509054749
DOIs
Publication statusPublished - Mar 2017
Event16th IEEE International Conference on Data Mining, ICDM 2016 - Barcelona, Catalonia, Spain
Duration: 12 Dec 201615 Dec 2016

Publication series

NameProceedings of the IEEE International Conference on Data Mining (ICDM)
PublisherIEEE
ISSN (Print)2375-8486

Conference

Conference16th IEEE International Conference on Data Mining, ICDM 2016
Country/TerritorySpain
CityBarcelona, Catalonia
Period12/12/1615/12/16

Structured keywords

  • Jean Golding
  • SPHERE

Keywords

  • Training data
  • Estimation
  • Standards
  • Reliability
  • Data models
  • Probability
  • Conferences
  • Multiclass classification
  • Outlier detection
  • Confident classification

Fingerprint

Dive into the research topics of 'Background Check: A General Technique to Build More Reliable and Versatile Classifiers'. Together they form a unique fingerprint.
  • HPC (High Performance Computing) Facility

    Susan L Pywell (Manager), Simon A Burbidge (Other), Polly E Eccleston (Other) & Simon H Atack (Other)

    Facility/equipment: Facility

Cite this