Research output per year
Research output per year
Meelis Kull, Peter A. Flach
Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)
We propose a general method to assess the reliability of two-class probabilities in an instance-wise manner. This is relevant, for instance, for obtaining calibrated multi-class probabilities from two-class probability scores. The LS-ECOC method approaches this by performing least-squares fitting over a suitable error-correcting output code matrix, where the optimisation resolves potential conflicts in the input probabilities. While this gives all input probabilities equal weight, we would like to spend less effort fitting unreliable probability estimates. We introduce the concept of a reliability map to accompany the more conventional notion of calibration map; and LS-ECOC-R which modifies LS-ECOC to take reliability into account. We demonstrate on synthetic data that this gets us closer to the Bayes-optimal classifier, even if the base classifiers are linear and hence have high bias. Results on UCI data sets demonstrate that multi-class accuracy also improves. © 2014 Springer-Verlag.
Original language | English |
---|---|
Title of host publication | Machine Learning and Knowledge Discovery in Databases |
Subtitle of host publication | European Conference, ECML PKDD 2014, Nancy, France, September 15-19, 2014. Proceedings, Part II |
Editors | Toon Calders, Floriana Esposito, Eyke Hullermeier, Rosa Meo |
Publisher | Springer Berlin Heidelberg |
Pages | 18-33 |
Number of pages | 16 |
ISBN (Electronic) | 9783662448519 |
ISBN (Print) | 9783662448502 |
DOIs | |
Publication status | Published - 1 Sept 2014 |
Event | European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014 - Nancy, France Duration: 15 Sept 2014 → 19 Sept 2014 |
Name | Lecture Notes in Artificial Intelligence |
---|---|
Publisher | Springer Berlin Heidelberg |
Volume | 8725 |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference | European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014 |
---|---|
Country/Territory | France |
City | Nancy |
Period | 15/09/14 → 19/09/14 |
Research output: Contribution to journal › Article (Academic Journal) › peer-review
Flach, P. A. (Principal Investigator)
Engineering and Physical Sciences Research Council
1/02/13 → 1/08/16
Project: Research
Flach, P. A. (Recipient), 2014
Prize: Prizes, Medals, Awards and Grants