How to select only those image folders containing images more than 100. From a set of 193 image folders.

Here i have 193 folders, where in each folder contains a set of images. In each folder, total number of images are not the same. It is like 15, 28, 64, 96, 125, 265, 456, 564, and so on. Now i need to take only those folders where my images count is more than 100.(out of these 193 folders). Then splitting it into training and testing. How i can do it? I tried this code. but its not working properly. Please help.

 SetDir  = fullfile('D:\MATLAB\Proj_Work\seperated');
Imds = imageDatastore(SetDir,'IncludeSubfolders',true,'LabelSource','foldernames');
T = countEachLabel(Imds);
minSetCount = 100;%min([T.Count]);
Imds_New = splitEachLabel(Imds,minSetCount,'randomize');
[trainingSet,testSet]=splitEachLabel(Imds_New,0.7,'randomize');

[Jan]

"not working properly" is not useful as description of the problem. It would be more helpful, if you explain, what the problem is, than if the readers guess this important detail. Please edit the question and append a description.

[Karthik K]

@jan. Not working means, i am not able to fetch only those folders which contains images more than 100 in it. Out of 193 folders only those folders where images contains more than 100 in it. i need only those to get it before i split for training and testing.

[Jan]

"not able to fetch" is not better than "not working". Please explain, what you observe.

[Karthik K]

@Jan. Say i have 193 folders, each folder contains images. In each folder, total number of images are not the same. It is like 15, 28, 64, 96, 125, 265, 456, 564, and so on. Now out of 193 folders only those folders where images contain more than 100 in it, only those folders i need to have for my work.(Project). Later i will split for training and testing. But now according to the above code i am not able to have only those folders where image count is equal or more than 100 in Imds_New (line number 5).

[Jan]

@karthik k: It is clear already, what you want to achieve. There is no need to repeat this explanation. You have posted some code. The open question is, what you observe, when you run it. We want to fix the problem with your code, but you do not mention, what the problem is. All we know yet is: "not able to fetch", "not working" and "not able to have only those folders". Do not describe, what the code doesn't do, but explain, what it does instead. This is the 3rd question for clarification concerning the same point.

I do not have the toolbox to run imageDatastore. It would be easy to write a code with a loop and some dir commands. But this might not match your current workflow.

[Karthik K]

I think there is a problem in my explanation to you sir. Line by line I will try to point it out.

SetDir  = fullfile('D:\MATLAB\Proj_Work\seperated'); % My Main folder path.

Imds = imageDatastore(SetDir,'IncludeSubfolders',true,'LabelSource','foldernames'); % All Sub folders with its labels. (193 folders, images in each folder along with its label)

T = countEachLabel(Imds); % Total count in each folder 

minSetCount = 100;%min([T.Count]); % tried to set it to 100. Which i need.

Imds_New = splitEachLabel(Imds,minSetCount,'randomize'); % Here i need to get only those folders where images is equal or greater than 100.

Instead i will end up with error, because there are folders with images less than 100 also. reason is minSetCount=100 does not come true here.

The error is

Error using matlab.io.datastore.ImageDatastore/splitEachLabel (line 205)
Sum of proportions (100) exceeds the minimum number of files (1) in all labels.

Use countEachLabel method to find the number of files in each label.

Error in Untitled (line 5)
Imds_New = splitEachLabel(Imds,minSetCount,'randomize');

Answer 1

[Image Analyst]

karthik, try this:

SetDir  = fullfile('D:\MATLAB\Proj_Work\seperated'); % My Main folder path.

imds = imageDatastore(SetDir,'IncludeSubfolders',true,'LabelSource','foldernames'); % All Sub folders with its labels. (193 folders, images in each folder along with its label)
% Get the total number of files in each folder.
T = countEachLabel(imds); % Total count in each folder

% See which folders have at least 100 files in them.
foldersWithEnoughFiles = T.Count >= 100;

% Build a list (cell array) of all the folders.
allFolderList = cell(size(T, 1), 1);
for k = 1 : size(T, 1)
  [thisFolder, ~, ~] = fileparts(imds.Files{k});
  allFolderList{k} = thisFolder;
end
% Extract only those folders with 100 files or more.
goodFolders = unique(allFolderList(foldersWithEnoughFiles));
% Get a new datastore with only those folders with 100 or more files in them.
imds_good = imageDatastore(goodFolders,'IncludeSubfolders',false,'LabelSource','foldernames'); % All Sub folders with its labels. (193 folders, images in each folder along with its label)
% Split all files into a test set and a training set.
splitPercentage = 0.8;
[imds_train, imds_test] = splitEachLabel(imds_good, splitPercentage, 'randomize');

[Karthik K]

Thank You very much Sir. (Image Analyst). With some light modification it is working now. But you made me think about it and change it. I appreciate you sir. Once again thankful to you.

I changed at for loop. instead size(T,1) i changed it to for k = 1 : size(imds.Files, 1). because it will read those first 193 images with its folder instead of all 193 folders. Then with unique folders and with a for loop those having more than 100 images is considered sir.