A Comparison of Different ROC Measures for Ordinal Regression.pdf
文本预览下载声明
A Comparison of Different ROC Measures for Ordinal Regression
Willem Waegeman Willem.Waegeman@UGent.be
Department of Electrical Energy, Systems and Automation, Ghent University, Technologiepark 913, B-9052
Ghent, Belgium
Bernard De Baets Bernard.DeBaets@UGent.be
Department of Applied Mathematics, Biometrics and Process Control, Ghent University, Coupure links 653,
B-9000 Ghent, Belgium
Luc Boullart Luc.Boullart@UGent.be
Department of Electrical Energy, Systems and Automation, Ghent University, Technologiepark 913, B-9052
Ghent, Belgium
Abstract
Ordinal regression learning has characteris-
tics of both multi-class classification and met-
ric regression because labels take ordered,
discrete values. In applications of ordinal re-
gression, the misclassification cost among the
classes often differs and with different mis-
classification costs the common performance
measures are not appropriate. Therefore we
extend ROC analysis principles to ordinal re-
gression. We derive an exact expression for
the volume under the ROC surface (VUS)
spanned by the true positive rates for each
class and show its interpretation as the prob-
ability that a randomly drawn sequence with
one object of each class is correctly ranked.
Because the computation of V US has a huge
time complexity, we also propose three ap-
proximations to this measure. Furthermore,
the properties of VUS and its relationship
with the approximations are analyzed by sim-
ulation. The results demonstrate that opti-
mizing various measures will lead to different
models.
1. Introduction
In multi-class classification labels are picked from a
set of unordered categories. In metric regression labels
might take continuous values. Ordinal regression can
Appearing in Proceedings of the ICML 2006 workshop on
ROC Analysis in Machine Learning, Pittsburgh, USA,
2006. Copyright 2006 by the author(s)/owner(s).
be located in between these learning problems because
here labels are chosen from a set of ordered categories.
Applications
显示全部