How can I pass datetime/timestamp as a filter value to Task History API Query?

Matillion ETL
1

0694G00000E350tQAB_0D54G00007fMTmgSAG
0694G00000E350tQAB_0D54G00007fMTmgSAG1067×155 14.7 KB

I am trying to load the task history data incrementally after each run of our jobs.

However, I am only able to pass a ‘Date’ value(stored as variable ${max_summar_date}) to the ‘since’ column to the field. Please see the screenshot.

${max_summar_date} is the high watermark value of enqueued time in the database.

I have tried using other columns such as enqueuedTime but it did not work either.

 

Any ideas on how to pass datetime/timestamp values?

2

Hi Sarin,

 

According to the Matillion API docs, <since> takes the form yyyy-MM-dd

 

So, I'd recommend trying first with a hardcoded value. Then if that works, switch to a Matillion variable. I tend to use the Text type rather than DateTime, since you have more control over the formatting that way.

 

Hope that helps!

 

BR

3

Thanks @deBaris​ for your suggestion. Yes, <since> takes in only yyyy-MM-dd format.

However, I found that the <id> column is useful to determine new job runs incrementally using high water mark method. I was able to cast the <Id> field as 'INT' and load only the latest task history since the previous load.

4

^^ nice, thanks for sharing!