Data transfer-component sometimes fails due to bad AWS credentials when target is a S3 bucket

Matillion ETL
1

I am using the Data transfer-component to load a file from Azure blob storage to an S3 Bucket. I have added the AWS credentials in the credentials manager.

 

Sometimes I get an error in the Data transfer component.

"Parameter Validation Failure:

Target URL - That URL does not point to an accessible S3 bucket."

 

This error coincides with the following error when I enter the Target URL field in the component:

The authorization header is malformed; the authorization component "Credential=S0xxxxxxxxxxxxxxxx1lMQ0c=/20210610/us-east-1/s3/aws4_request" is malformed.

(Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: P3MKxxxxxxxxxNZMP1; S3 Extended Request ID: 2V0P7xxxxxxxxxxxxxxly3bS+s/TRDKxxxxxxxxxUg+dmLttNo1xxxxxxxxxxxZki1nLI6JY=; Proxy: null)

 

I am having a hard time reproducing the error. Anyone have a clue what is wrong her.

 

 

I'm on an Azure hosted Matillion instance, with snowflake as the database.

2

I have got the same error on S3 Load component whenever the Matillion instance is rebooted .

"Parameter Validation Failure:

Target URL - That URL does not point to an accessible S3 bucket."

 

The credential in the error message of the Target URL field starts with 'S0' while my AWS Key ID starts with 'AKI'.

 

I am on an AWS hosted Matillion instance.

 

when i get the error, i have to go to 'Manage Credentials' and click 'Test' button. Then come back to the S3 bucket, and the error is gone .

 

Any suggestion?

3

It sounds like we have the exact same problem. I have confirmed that a reboot of the machine makes the problem appear. I currently have no suggestion on how to proceed. Is it possible to trigger a test of the credentials in the job with some scripting maybe?

4

Recently we migrated to the latest version.We are also receiving same error.

Please give us some advice!

@JoeCommunityManager​

5

What version did you migrate to? According to Matillion support, this should be fixed in 1.54.

6

i see the issue in versions 1.51 and 1.53.10. when will be 1.54 available?

7

update on 06/29/2021: I have upgraded to 1.54.9 and I still see the issue

8

Hi all,

 

There was a bug related to this that was fixed in version 1.56 if you upgrade to this version, this should be fixed.

 

Kind regards, Joe

9

I was told the following by a Matillion support person on July 5th

 

"July 5, 2021 at 11:34 AM

 

 

Hi,

Apologies for this, this issue was fixed in version 1.54.0 but the issue regressed in version 1.54.9, we have raised this internally with engineering and its been worked on. I will update you as soon an update is available"

 

Have not heard back.