# S3 bucket/files for training not public?
**URL:** https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397
**Category:** Matillion ETL
**Created:** [August 4, 2021, 9:11am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397 "2021-08-04T09:11:30Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@frenke](https://matillioncommunity.discourse.group/u/frenke)
#### Post date: [August 4, 2021, 9:11am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/1 "2021-08-04T09:11:30Z")
Hello... very quick question...
While following the training steps provided in this page [LINK](https://academy.matillion.com/guides/33d32ffe-aef3-11ea-adeb-06c0361096e5/tasks/350edfbc-aef3-11ea-9bfa-0667248b3975?certification_id=0baba768-b088-11ea-b5e9-063e41907789&certification_id=0baba768-b088-11ea-b5e9-063e41907789) .. I'm not able to read the S3 bucket where the files I need are.
The bucket is not accessible but according to the description of the task it should be:
You will be reading a compressed CSV file located in a public S3 bucket located in our AWS account.
This is the s3 bucket: **s3://online-mtln-training-s3-flights/**
Did some of you had the same issue? If so and this is a problem on matillion side how can I ask them to check?
Thanks for now
F.
PS just in case..... yes, I've noticed that the S3 bucket in the documentation and in their screenshot is different (in the screenshot they use s3://mtln-training-s3-flights/ ). Also this one is not public.
---
### Author:  [@ian.funnell](https://matillioncommunity.discourse.group/u/ian.funnell)
#### Post date: [August 4, 2021, 11:46am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/2 "2021-08-04T11:46:20Z")
Hi [@frenke](https://matillioncommunity.discourse.group/u/frenke),
I was wondering if the root cause is the S3 privilege you have on your Matillion ETL instance.
If you press the [Test button](https://documentation.matillion.com/docs/9184923) in your Environment, to verify the cloud API privileges, what do you see?
Something like this is good...
Whereas something like this is bad...
If you are seeing a "check credentials" warning, I would guess you are missing some of the S3 privileges. We have [this document](https://documentation.matillion.com/docs/2954248) on various ways to set up IAM Roles & Permissions on AWS, which should help you add them.
Ian
---
### Author:  [@frenke](https://matillioncommunity.discourse.group/u/frenke)
#### Post date: [August 4, 2021, 2:04pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/3 "2021-08-04T14:04:34Z")
Hi Ian, nice to meet you and thanks for the answer.
I have indeed the warning for S3. That is strange because I have full access (allow all actions to a dedicated bucket).
I played a little but with permissions and while there's still something I don't understand or I don't like I was able to have the S3 API:success 🙌 ....... and more specifically read the external bucket for the training.
**So you are right, that was the root cause.** 🍾
Thanks again for the help!
F
---
### Author:  [@frenke](https://matillioncommunity.discourse.group/u/frenke)
#### Post date: [August 5, 2021, 9:16am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/4 "2021-08-05T09:16:53Z")
Hello [@Bryan](https://matillioncommunity.discourse.group/u/Bryan) , nice to meet you and thanks for your reply
I agree with what you say, with roles we have so much more control and understanding, thanks!
Just to add some more information to the initial problem (maybe also [@ian.funnell](https://matillioncommunity.discourse.group/u/ian.funnell) could be interested?) ... I already had full permissions on my bucket.
The problem that I found is that the initial policy was something like ALLOW ALL ACTIONS ON BUCKET _XYZ_ AND OBJECTS _XYZ_/\*
there are indeed actions/permissions that are generic and they don't target buckets/objects, one of these permissions is the "ListAllMyBuckets" that needs to have the ALLOW on ALL S3 resources.
Our policy was generated with a script so we thought allowing ALL on everything in that bucket was enough but it is not, we need to grant "ListAllMyBuckets" to the overall S3 service.
As stated in the docs this is used for exploration and discovery so it is not strictly needed, but I wanted to see the "_S3 API:success_" so for the moment I'm ok with that.
lesson learnt in this case, remember what I already knew that some permissions needs to be applied on the overall service, not specific objects/buckets.
Thanks again for the help and support, I'm back in business for the moment.
FL
---
### Author:  [@AL171878](https://matillioncommunity.discourse.group/u/AL171878)
#### Post date: [September 9, 2021, 6:40pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/5 "2021-09-09T18:40:16Z")
I have same issue but with different message, it says that the S3 bucket does not exists
instead of that I can see other folders but not the one with files from Matillion Academy training
---
### Author:  [@Bryan](https://matillioncommunity.discourse.group/u/Bryan)
#### Post date: [August 4, 2021, 2:37pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/6 "2021-08-04T14:37:58Z")
As a follow-up [@frenke](https://matillioncommunity.discourse.group/u/frenke),
Check that your IAM role that Matillion is using has permissions to that particular S3 bucket. The proper way to control permissions in AWS is through IAM roles and not the services or objects themselves. I have seen where people mix IAM role permissions with service/object (in your case S3 bucket) permissions. Although you can do this, it becomes extremely hard to understand what the role and/or users in that role have permissions to.
---
### Author:  [@david.foster](https://matillioncommunity.discourse.group/u/david.foster)
#### Post date: [April 19, 2024, 9:22am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/7 "2024-04-19T09:22:06Z")
I am having a similiar issue trying to read the Airport avro files in AWS S3.
Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS\_ACCESS\_KEY\_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS\_ACCESS\_KEY\_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS\_WEB\_IDENTITY\_TOKEN\_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])): Profile file contained no credentials for profile 'default': ProfileFile(profilesAndSectionsMap=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS\_CONTAINER\_CREDENTIALS\_FULL\_URI or AWS\_CONTAINER\_CREDENTIALS\_RELATIVE\_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
The AWS connection test for S3 is successful, the test does say 'Check Credentials' for DMS (dont know if this is relevant.
Any help would be appreciated.
---
### Author:  [@drock1635272309580](https://matillioncommunity.discourse.group/u/drock1635272309580)
#### Post date: [October 26, 2021, 6:48pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/8 "2021-10-26T18:48:10Z")
I am currently having this same issue - getting Status Code: 404; Error Code: NoSuchBucket;
Has anyone responded with a fix?
---
### Author:  [@david.foster](https://matillioncommunity.discourse.group/u/david.foster)
#### Post date: [April 22, 2024, 1:20pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/9 "2024-04-22T13:20:20Z")
Resolved this issue. The environment settings for AWS by default is 'Instance'. I had setup an AWS IAM user and group. You can add and test this user in the AWS user wizard, but it did not switch over the actual setting. Once I noticed the setting was still on 'Instance' I changed it to the AWS IAM user and the S3 connection to the sample data worked.
---
### Author:  [@AL171878](https://matillioncommunity.discourse.group/u/AL171878)
#### Post date: [October 27, 2021, 1:13pm UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/10 "2021-10-27T13:13:47Z")
I resolved my self, I downloaded the .gz files from Google repository and copy those into a S3 bucket, then I just make reference to where I placed the files, and that was it.
---
### Author:  [@ClaireSeniorCommunityManager](https://matillioncommunity.discourse.group/u/ClaireSeniorCommunityManager)
#### Post date: [May 14, 2024, 8:05am UTC](https://matillioncommunity.discourse.group/t/s3-bucket-files-for-training-not-public/1397/11 "2024-05-14T08:05:12Z")
Hi [@david.foster](https://matillioncommunity.discourse.group/u/david.foster)! Thanks so much for returning to share how you solved the issue.
I hope to see you around the forums again!
Claire