USing

Matillion ETL
1

Can we capture job start and end time and row count using job variables.

 

Assigning start_time , row_count to variables and inserting to snowflake table.

but job values are not being loaded . I see nulls in snowflake table.

 

Any help?

 

Thanks.

 

 

 

2

Hi if you expect this for audit purpose you can make sql script to update before start and end time after success.​

​

For row count i found this in forum please check if it helps ​https://matillioncommunity.discourse.group/t/hi-guys-what-tools-in-matillion-can-i-use-get-a-row-count-of-all-the-tables-in-a-snowflake-schema-without-using-the-information-schema/1579

3

The best way to get this audit information would be exploring Task History API. Matillion already records Start Time, End Time, Duration and Row Count and Message information for every task executed in its Postgres Database.

 

Calling TaskHistory API and recording the data into a Snowflake table would be an easier option , I reckon.

 

The Other possible way is to use a Shared Job which fetches and logs the values into a Table. And every Orchestration/Transformation job needs to be run via this Shared Job.