Improving the AUC of Probabilistic Estimation Trees

Ferri Cesar; Peter Flach; Hernandez-Orallo Jose

Improving the AUC of Probabilistic Estimation Trees

Ferri Cesar, Peter Flach, Hernandez-Orallo Jose

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

50 Citations (Scopus)

Abstract

In this work we investigate several issues in order to improve the per- formance of probabilistic estimation trees (PETs). First, we derive a new prob- ability smoothing that takes into account the class distributions of all the nodes from the root to each leaf. Secondly, we introduce or adapt some new splitting criteria aimed at improving probability estimates rather than improving classifi- cation accuracy, and compare them with other accuracy-aimed splitting criteria. Thirdly, we analyse the effect of pruning methods and we choose a cardinality- based pruning, which is able to significantly reduce the size of the trees without degrading the quality of the estimates. The quality of probability estimates of these three issues is evaluated by the 1-vs-1 multi-class extension of the Area Under the ROC Curve (AUC) measure, which is becoming widespread for evaluating probability estimators, ranking of predictions in particular.

Translated title of the contribution	Improving the AUC of Probabilistic Estimation Trees
Original language	English
Title of host publication	Proceedings of the 14th European Conference on Machine Learning
Pages	121-132
Publication status	Published - 2003

Bibliographical note

ISBN: 3540201211
Publisher: Springer-Verlag
Name and Venue of Conference: Proceedings of the 14th European Conference on Machine Learning
Other identifier: 2000549

Access to Document

http://www.cs.bris.ac.uk/Publications/pub_master.jsp?id=2000549

Cite this

@inproceedings{2896b5eda3c0425ca2e5d5203b5ba491,

title = "Improving the AUC of Probabilistic Estimation Trees",

abstract = "In this work we investigate several issues in order to improve the per- formance of probabilistic estimation trees (PETs). First, we derive a new prob- ability smoothing that takes into account the class distributions of all the nodes from the root to each leaf. Secondly, we introduce or adapt some new splitting criteria aimed at improving probability estimates rather than improving classifi- cation accuracy, and compare them with other accuracy-aimed splitting criteria. Thirdly, we analyse the effect of pruning methods and we choose a cardinality- based pruning, which is able to significantly reduce the size of the trees without degrading the quality of the estimates. The quality of probability estimates of these three issues is evaluated by the 1-vs-1 multi-class extension of the Area Under the ROC Curve (AUC) measure, which is becoming widespread for evaluating probability estimators, ranking of predictions in particular. ",

author = "Ferri Cesar and Peter Flach and Hernandez-Orallo Jose",

note = "ISBN: 3540201211 Publisher: Springer-Verlag Name and Venue of Conference: Proceedings of the 14th European Conference on Machine Learning Other identifier: 2000549",

year = "2003",

language = "English",

pages = "121--132",

booktitle = "Proceedings of the 14th European Conference on Machine Learning",

}

TY - GEN

T1 - Improving the AUC of Probabilistic Estimation Trees

AU - Cesar, Ferri

AU - Flach, Peter

AU - Jose, Hernandez-Orallo

N1 - ISBN: 3540201211 Publisher: Springer-Verlag Name and Venue of Conference: Proceedings of the 14th European Conference on Machine Learning Other identifier: 2000549

PY - 2003

Y1 - 2003

N2 - In this work we investigate several issues in order to improve the per- formance of probabilistic estimation trees (PETs). First, we derive a new prob- ability smoothing that takes into account the class distributions of all the nodes from the root to each leaf. Secondly, we introduce or adapt some new splitting criteria aimed at improving probability estimates rather than improving classifi- cation accuracy, and compare them with other accuracy-aimed splitting criteria. Thirdly, we analyse the effect of pruning methods and we choose a cardinality- based pruning, which is able to significantly reduce the size of the trees without degrading the quality of the estimates. The quality of probability estimates of these three issues is evaluated by the 1-vs-1 multi-class extension of the Area Under the ROC Curve (AUC) measure, which is becoming widespread for evaluating probability estimators, ranking of predictions in particular.

AB - In this work we investigate several issues in order to improve the per- formance of probabilistic estimation trees (PETs). First, we derive a new prob- ability smoothing that takes into account the class distributions of all the nodes from the root to each leaf. Secondly, we introduce or adapt some new splitting criteria aimed at improving probability estimates rather than improving classifi- cation accuracy, and compare them with other accuracy-aimed splitting criteria. Thirdly, we analyse the effect of pruning methods and we choose a cardinality- based pruning, which is able to significantly reduce the size of the trees without degrading the quality of the estimates. The quality of probability estimates of these three issues is evaluated by the 1-vs-1 multi-class extension of the Area Under the ROC Curve (AUC) measure, which is becoming widespread for evaluating probability estimators, ranking of predictions in particular.

M3 - Conference Contribution (Conference Proceeding)

SP - 121

EP - 132

BT - Proceedings of the 14th European Conference on Machine Learning

ER -