Understanding accuracyValidation Output of Classifier

I've been working with the Classification Application and generated functions for 3 different tree-based classification algorithms. The data set consists of three continuous variables and one classification column where the data can be one of three states: NL, MG and LR.

Reading the comments in the automatically generated code, it describes the accuracyValidation output as:

A double containing the accuracy in percent. In the app, the History list displays this overall accuracy score for each model.

However, when I output it, it is a 3 column binary matrix so I'm not sure how to interpret it. Can someone help by explaining this? I have included the training and testing datasets alongside the main file. Many thanks.

Answer 1

[Pratyush Roy]

Hi Josh,

The output acc is obtained by running the function predictFcn on the tree model.

[CoarseTreeY,acc] = trainedCoarseTreeModel.predictFcn(testData); %line 18

The predictFcn uses the predict function from the Statistics and Machine Learning toolbox. This function is used to predict the labels and the posterior scores for each class. The posterior scores are returned as a N*K matrix, N being the number of instances of test data and K being the number of classes. The value of K is 3 in this case, as a result a N*3 matrix is getting created. The following snippet shows the use of predict function in the code:

predictorExtractionFcn = @(t) t(:, predictorNames); % line 95
treePredictFcn = @(x) predict(classificationTree, x); % line 96
trainedClassifier.predictFcn = @(x) treePredictFcn(predictorExtractionFcn(x)); % line 97

This is not the same as validation accuracy which is returned by calling the trainCoarseTreeClassifier function.

The following snippet might be used to obtain the validation accuracy as per the description:

[trainedFineTreeModel,validationAccuracy] = trainCoarseTreeClassifier(trainData);

Hope this helps!