An analysis of rule evaluation metrics

J Furnkranz, PA Flach

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

64 Citations (Scopus)

Abstract

In this paper we analyze the most popular evaluation metrics for separate-and-conquer rule learning algorithms. Our results show that all commonly used heuristics, including accuracy, weighted relative accuracy, entropy, Gini index and information gain, are equivalent to one of two fundamental prototypes: precision, which tries to optimize the area under the ROC curve for unknown costs, and a cost-weighted difference between covered positive and negative examples, which tries to find the optimal point under known or assumed costs. We also show that a straight-forward generalization of the m-estimate trades off these two prototypes.
Translated title of the contributionAn analysis of rule evaluation metrics
Original languageEnglish
Title of host publicationUnknown
PublisherAAAI Press
Pages202 - 209
Number of pages7
ISBN (Print)1577351894
Publication statusPublished - Jan 2003

Bibliographical note

Conference Proceedings/Title of Journal: Proc. 20th International Conference on Machine Learning (ICML'03)

Fingerprint Dive into the research topics of 'An analysis of rule evaluation metrics'. Together they form a unique fingerprint.

Cite this