Comparing the classification results ?

I want to compare the result of ANN with some other classifiers such as LDA, SVM , etc. and I am using 10-fold cross validation (9 for training and 1 for testing) for those classifiers. In ANN, the data is divided to three parts (0.6 training 0.2 validation 0.2 testing). How can I evaluate all the methods in the same way. For example 10-fold cross validation for all of them including ANN.

Answer 1

[Greg Heath]

If the other classifiers do not use a validation set, then you should not either. In fact you should use the exact same training and testing subsets.

So, change your default 0.7/0.15/0.5 ratio to 0.9/0.0/0.1 and use the exact same data divisions for all classifiers.

The MATLAB default is PATTERNNET (with TRAINSCG).

If NN results are bad, post again.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Greg Heath]

CORRECTION:

 default trn/val/tst ratios are 0.7/0.15/0.15

Greg