Predictive Performance of Weighted Relative Accuracy

Weighted relative accuracy was proposed in iteilp99-lavrac-flach-zupan as an alternative to classification accuracy typically used in inductive rule learners. Weighted relative accuracy takes into account the improvement of the accuracy relative to the default rule (i.e., the rule stating that the same class should be assigned to all examples), and also explicitly incorporates the generality of a rule (i.e., the number of examples covered). In order to measure the predictive performance of weighted relative accuracy, we implemented it in the rule induction algorithm CN2. Our main results are that weighted relative accuracy dramatically reduces the size of the rule sets induced with CN2 (on average by a factor 9 on the 23 datasets we used), at the expense of only a small average drop in classification accuracy.