Receiver Operating Characteristics (ROC) Analysis originated from signal detection theory, as a model of how well a receiver is able to detect a signal in the presence of noise. Its key feature is the distinction between hit rate (or true positive rate) and false alarm rate (or false positive rate) as two separate performance measures. It has been introduced to machine learning relatively recently, in response to classification tasks with skewed class distributions or misclassification costs. ROC analysis is set to cause a paradigm shift in classification-oriented machine learning. Separating performance on classes can help us understand the behaviour and skew-sensitivity of many machine learning metrics by plotting their isometrics in ROC space; develop methods and algorithms to improve the Area Under the ROC Curve (AUC) of a model; understand fundamental classifier training algorithms by visualising how they split up ROC space or its unnormalised cousin, PN space. In this talk I will mainly illustrate the latter point.
|Translated title of the contribution||Reinventing Machine Learning with ROC Analysis (invited talk)|
|Title of host publication||Proceedings of the 10th Ibero-American Conference on AI and the 18th Brazilian AI Symposium|
|Publication status||Published - 2006|
Bibliographical noteISBN: 354045462
Name and Venue of Conference: Proceedings of the 10th Ibero-American Conference on AI and the 18th Brazilian AI Symposium
Other identifier: 2000625