Doubts about cross-validation

1 visualización (últimos 30 días)
Elena Casiraghi
Elena Casiraghi el 18 de Ag. de 2019
Comentada: Elena Casiraghi el 22 de Ag. de 2019
I'm trying to test performance of a KNN and a svm (cecoc) classifier for a multiclass problem.
For the knn model I do it by building mdlknn, which is a cross validated model (leave one out) and then using kfoldLoss function
mdlknn = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel');
perf = 1-kfoldLoss(mdlknn);
Is this correct? My doubts regards the cross-bvalidation:
Using option ('Leaveout','on') in fitcknn is the same as not using it and then calling crossval on the trained model?
mdlknnNoCrossVal = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'HyperparameterOptimizationOptions','UseParallel');
mdlknn = crossval(mdlknnNoCrossVal, 'leaveout','on')
perf = 1-kfoldLoss(mdlknn);
Next, since in knn classifier I'm using my distance, I would like to do the same with the cecoc. How can I do that?
At this stage, the code I use for building the cecoc doesn't use it. how can I set it?
t = templateSVM('Standardize',true,'KernelFunction','rbf');
mdlsvm = fitcecoc(X,labels, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel',...
'coding', 'ternarycomplete','Learners',t);
perf = 1-kfoldLoss(mdlsvm);
And again: Using option ('Leaveout','on') in fitcecoc is the same as not using it and then calling crossval on the trained model, right?
THANKS A LOT

Respuesta aceptada

Jyothis Gireesh
Jyothis Gireesh el 22 de Ag. de 2019
I am assuming that you want to have some clarification regarding the cross-validation syntax in KNN and SVM
  • KNN Classifier
mdlknn = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel');
mdlknnNoCrossVal = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'HyperparameterOptimizationOptions','UseParallel');
mdlknn = crossval(mdlknnNoCrossVal, 'leaveout','on')
These two syntaxes may be equivalent for the creation of a cross-validated KNN model. “mdlknn" is a "ClassificationPartitionModel" classifier. "mdlknnNoCrossVal” creates a "ClassificationKNN" classifier which is cross validated using the “crossval” function.
  • ECOC classifier
In the case of ECOC classifier you may use the templateKNN as the learner. It provides the feature to customize the distance metric using the ‘Distance’ argument. The distance metric can be passed as a function handle like that of the KNN classifier syntax.
And the two syntaxes are valid for the process of cross validation in the case of ECOC also.
  1 comentario
Elena Casiraghi
Elena Casiraghi el 22 de Ag. de 2019
Thanks a lot!
this clarified what I was asking!

Iniciar sesión para comentar.

Más respuestas (0)

Etiquetas

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by