LaCova: A Tree-Based Multi-Label Classifier using Label Covariance as Splitting Criterion

Dealing with multiple labels is a supervised learning problem of increasing importance. Multi-label classifiers face the challenge of exploiting correlations between labels. While in existing work these correlations are often modelled globally, in this paper we use the divide-and-conquer approach of decision trees which enables taking local decisions about how best to model label dependency. The resulting algorithm establishes a tree-based multi-label classifier called LaCova which dynamically interpolates between two well-known baseline methods: Binary Relevance, which assumes all labels independent, and Label Powerset, which learns the joint label distribution. The key idea is a splitting criterion based on the label covariance matrix at that node, which allows us to choose between a horizontal split (branching on a feature) and a vertical split (separating the labels). Empirical results on 12 data sets show strong performance of the proposed method, particularly on data sets with hundreds of labels.