fileDatastore request to AWS S3 limited to 1000 files
4 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I have a bucket on an AWS S3 server which I can reach without any problems.
The issue is: The answer from the server is limited to 1000 files. How can I increase this number? My target is to get just the list of filenames.
I also tried different calls of fileDatastore.
imds = fileDatastore(['s3://',<bucket>,','/'],'FileExtensions','.csv','ReadFcn',@load,'IncludeSubfolders',true)
4 comentarios
Sameer Gummuluru
el 1 de Dic. de 2020
I was able to figure out a temporary solution to this, today. On older versions of MATLAB, I was able to use filedatastore to access more than 1000 files from S3. (I tried on MATLAB R2019a)
You can give it a try if working on older MATLAB versions is fine by you.
Jhon Wine
el 18 de Abr. de 2021
I have the same issue. When using Matlab R2020b I cannot get more than 1,000 files.
However while using Matlab R2019b, fileDataStore seems to return all the files I need.
Using AWS CLI seems to work fine, but is not as elegent of a solution.
Respuestas (1)
Div Tiwari
el 19 de Abr. de 2021
Hi all, this issue was resolved in MATLAB R2021a and R2020b Update 5: https://www.mathworks.com/support/bugreports/details/2407120.
If you're using R2020b, you can install updates by clicking on the notification icon towards the top-right of the MATLAB window.
1 comentario
Jhon Wine
el 22 de Abr. de 2021
I just installed R2021a, problem persists.
In fact I have two versions of Matlab on the same computer. 2019a shows 2000 files, 2021a shows 1000 files...
Ver también
Categorías
Más información sobre Introduction to Installation and Licensing en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!