Unable to access AWS: "Unable to find remote file. Ensure file exists and path is valid"

Unable to read from S3 from local docker container.

e.g.

>> fileread('s3://{BUCKET_NAME}/{PATH_TO_FILE}')

Error using fileread (line 23)

Could not open file s3://{BUCKET_NAME}/{FILE_PATH}.json. Unable to find remote file. Ensure file exists and path is valid..

I have bound a valid AWS credentials and config file to ~/.aws. I have also set all environment variables too:

setenv('AWS_ACCESS_KEY_ID', '******')

setenv('AWS_SECRET_ACCESS_KEY', '******')

setenv('AWS_DEFAULT_REGION', 'us-east-1')

setenv('AWS_SESSION_TOKEN','******')

setenv('AWS_SHARED_CREDENTIALS_FILE', '/root/.aws/credentials')

I am able to use system command to interact with S3:

>> system '~/.local/bin/aws s3 ls {BUCKET_NAME}'

2022-12-09 11:10:37 4204 *******.json

2022-12-13 15:10:57 4203 *******.json

2022-12-13 15:10:57 216790 *******.csv

2022-12-13 15:10:57 216786 *******.csv

Please help! NB: this is only an issue when running locally, the code works when running on an EC2 instance... Im thinking could it be a certificate issue? Is there a way I can force matlab to use installed certificates?

Answer 1

[Swastik Sarkar]

Hi @bc,

The webread  function can be used to access files stored in an S3 bucket. To access a file from an S3 bucket, the webread function can be utilized as follows:

URL = "https://s3.amazonaws.com/{BUCKET_NAME}/{PATH_TO_FILE}"; 
options = weboptions('ContentType', 'text'); 
data = webread(URL, options); 

For more information on using webread, refer to the below documentation:

https://www.mathworks.com/help/matlab/ref/webread.html