Hi, I am using a file iterator component, in the Filter regex parameter, I want to use environmental variable something like '.*`${testdt}`\d{6}.csv.gpg where testdt will resolve to a date value. The above is not working as testdt is not resolving

Matillion ETL
1

Hi,

I am using a file iterator component, in the Filter regex parameter, I want to use environmental variable something like '.*${testdt}\d{6}.csv.gpg where testdt will resolve to a date value. The above is not working as testdt is not resolving.

2

Hello @RK​

Looks like our community users have been unable to figure this one out, I would recommend raising a support ticket with our team and they can look into this for you. Any issues do let me know, I look forward to hearing how you work this out, please do let us know.

Thanks Joe

3

I am also facing the same issue.. was there any resolution for this?

4

Hello, this seems to be working quite fine for me atleast, please explain if you require something different:

 

I have S3 File Iterator pointed to a folder where I have 4 files:

 

With File Prefix as ".*" it results all 4:

Filename = test.json

Filename = Snowcastle_2020-10-19_11:00:00.json

Filename = Snowcastle_2020-10-19_11:00:0.json

Filename = Snowcastle_2020-10-19_11:00.json

 

I created a JOB VARIABLE name 'pvm' with value '2020-10-19' (string)

I applied File Prefix as following:

.*${pvm}_11:00.json

 

Result is:

Filename = Snowcastle_2020-10-19_11:00.json

 

 

What Input Data Type are you working with?

 

I believe you should handle prefix as string as-is, are you trying to format 6 digits from testdt only? Maybe put that handling in some other step than file iterator..?

 

 

5

Hi Michael, Thanks.. the example helped.!