Downloading a zipped file from S3 via AWS

15 visualizaciones (últimos 30 días)
John Spicer
John Spicer el 18 de Jul. de 2019
Respondida: Jingyu Si el 4 de Dic. de 2020
We're trying to download files (one at a time) from AWS S3.
I've used the example code I found on the web to get the list of files (works great), and I am trying to use this code to fetch the file:
fp= ['s3://ourBucketGoesHere/‘ selectedFile];
ds=fileDatastore(fp,'ReadFcn',@AWSRead, 'FileExtensions', {'.zip','','.txt'});
mydata=ds.read;
saveName = [filename ext];
save (saveName, 'mydata');
(also use this function for actual fetching) ->
function data = AWSRead(fileName)
fid = fopen(fileName);
data= fread(fid,inf);
fclose(fid);
end
Which does fetch the file.
The problem is that I can't save it in a format recognized. It should be gzip format (as it is coming from the server) but I can't make it work.
It's probably a setting I have wrong, I expect. Searching the net didn't find anything.

Respuestas (1)

Jingyu Si
Jingyu Si el 4 de Dic. de 2020
Try this funciton
ds=fileDatastore(fp,'ReadFcn',@S3_download);
dataFile=read(ds);
function dataFile=S3_download(readFileName)
writeFileName=regexp(readFileName,'\','split');
writeFileName=writeFileName{end}
readFileId = fopen(readFileName, 'r');
assert(readFileId > 0);
writeFileId = fopen(writeFileName, 'w');
assert(writeFileId > 0);
%%
buffersize = 1024;
while ~feof(readFileId)
fileData = fread(readFileId, buffersize, '*uint8');
writeCount = fwrite(writeFileId, fileData, 'uint8');
end
dataFile= writeFileName;
end

Categorías

Más información sobre Downloads en Help Center y File Exchange.

Etiquetas

Productos

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by