When executing the job, alert me if it does not find any files in my google storage

Matillion ETL
1

Hello,

 

I put the sales files every day in my google storage bucket. I created a variable to select multiple files to load.

 

And my job is scheduled to run every day at 8 a.m.

 

Is it possible on Matillion to configure a kind of alert if the job is executed and there are no files in my google storage to warn me ?

 

Thanks

 

 

2

Hi @tatus_parker​,

One way to do this is to use a Python script to check the count of files before your load starts. Take the count and set a Job Variable with that count. Next use an "If" component to determine the path towards loading or logging an error/message/etc based on the count.

I hope this helps!

3

I work on AWS S3. But, i believe, the idea could be the same.

 

Set a job variable, numeric, private, shared - to 0

Start --> File Iterator --> iterate to 'Python component with, context.updateVariable('COUNTER', '1') --> if component leading to a SNS Alert, with the condition 'COUNTER' is Equal to 0.

i will put that as a shared job, with the directory location as input parameter, so that i can customize the logic as elaborate as possible , like dont send SNS, if it was sent within last 2 hours or for the day, sending it to different SNS queues.