Downloading a zipped file from S3 via AWS
9 views (last 30 days)
Show older comments
We're trying to download files (one at a time) from AWS S3.
I've used the example code I found on the web to get the list of files (works great), and I am trying to use this code to fetch the file:
fp= ['s3://ourBucketGoesHere/‘ selectedFile];
ds=fileDatastore(fp,'ReadFcn',@AWSRead, 'FileExtensions', {'.zip','','.txt'});
mydata=ds.read;
saveName = [filename ext];
save (saveName, 'mydata');
(also use this function for actual fetching) ->
function data = AWSRead(fileName)
fid = fopen(fileName);
data= fread(fid,inf);
fclose(fid);
end
Which does fetch the file.
The problem is that I can't save it in a format recognized. It should be gzip format (as it is coming from the server) but I can't make it work.
It's probably a setting I have wrong, I expect. Searching the net didn't find anything.
0 Comments
Answers (1)
Jingyu Si
on 4 Dec 2020
Try this funciton
ds=fileDatastore(fp,'ReadFcn',@S3_download);
dataFile=read(ds);
function dataFile=S3_download(readFileName)
writeFileName=regexp(readFileName,'\','split');
writeFileName=writeFileName{end}
readFileId = fopen(readFileName, 'r');
assert(readFileId > 0);
writeFileId = fopen(writeFileName, 'w');
assert(writeFileId > 0);
%%
buffersize = 1024;
while ~feof(readFileId)
fileData = fread(readFileId, buffersize, '*uint8');
writeCount = fwrite(writeFileId, fileData, 'uint8');
end
dataFile= writeFileName;
end
0 Comments
See Also
Categories
Find more on Downloads in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!