ROC Curve with 3 classes

Hello,

I have a data set that looks like this:

the classifier can classify 3 classes (1,2,3). There are 300 values in each of the predictions (machine) and 300 values in the targets (manual) How can I turn it into a ROC curve in Matlab with all 3 classes?

Thanks in advance

Answer 1

[the cyclist]

Extending the concept of the ROC curve from a binary classification to multiclass classification is non-trivial. I had never heard of someone trying to do it before.

I found this paper by Hand & Till, which is implemented in this R package. The documentation of that package states, "A multiclass AUC is a mean of several auc and cannot be plotted."

You'll need to think carefully about what you are trying to learn from such an ROC curve. Alternative that might still be informative are pairwise comparison of each class, or one class vs. the other two.