I have close to 1 million images I'd like to prepare with imageDatastore (imds) for CNN deep learning training. I would like to create the imds via giving a cell array of the specific image paths.
These Paths are split among 1000s of directories or so. When creating a sub-set as a test, say around 30,000 images, it takes imageDataStore around 30sec or so to initialize. If I try all 1 million, it was taking longer than 2 hours! I did not wait for it to finish. I checked the code profiler to see what the issue was and it turned out to be that it was always checking the dir of each image. (see image)
My alternative was to simply not use imds all together and load 160GB would of images into memory!
Suggestions please. Is there a way to disable this directory checking mechanism?