When use K-cross validation concept to classify images, is the dataset is divided for training and testing? or for used for training and validation?

When use K-cross validation concept in pretrained CNN model to classify images, the dataset is split to k folds for example folde=5. Which means the for each fold we have 80% of the dataset used for training the model.

My question is about the rest 20% of the dataset in each fold. Is this rest part of the dataset is used for testing (to evaluate the classifier) in each fold? or it is used to validate the algorithm in each fold?

Actually I asked this question because some of researchers use K- cross validation to train (k-1) parts of the dataset and the rest part to validate the algorithm in each fold. while other researchers use the K- cross validation to test rest part of the dataset in each fold.

I need to use K-cross validation in my research but I confused about using this dataset part. Shall I use it for testing in each fold or use it to validate the algorithm?

As you know there is a big difference between validate dataset and testing dataset.

Thank you very Much

Your Rapid response is highly appreciated

Answer 1

[Tejas]

Hello Fatemah,

When using machine learning technique like K-Cross Validation, the best practice is to:

• Start by splitting the data into training and test sets.
• Divide the training data into K folds.
• In each iteration, use K-1 folds to train the model and the remaining fold to validate it.
• After this process, a model with the best parameters is obtained, which can then be tested using the test set.

This approach ensures the model is evaluated on unseen data, providing a better assessment of its performance.

Alternate approach is to use the entire dataset as training data:

• Split the entire dataset into K folds, without a separate test set.
• Again, use K-1 folds for training and the remaining fold for validation.
• Here, since there is no separate test set, the validation fold in each iteration serves as the test fold, and the validation accuracy is used to evaluate the model.

Refer to below links for more information on:

• K-Cross Validation: https://www.analyticsvidhya.com/blog/2022/02/k-fold-cross-validation-technique-and-its-essentials/
• Implementation of K-Cross Validation in MATLAB: https://www.mathworks.com/support/search.html/answers/2149929-how-to-divide-image-datastore-into-training-set-validation-set-and-test-set-for-training-a-cnn-netw.html