Is there a way to iterate and execute all orchestration jobs in a folder?

Matillion ETL
1

Within an Orchestration Job, I'd like to loop over all jobs in a folder and execute them.

 

I am developing a test suite and want to automatically call these jobs which would generate test data / validate data after a pipeline run.

An alternative could be some sort of naming convention with a prefix (e.g. run all jobs matching test_*).

 

Is there a way to achieve this?

Thanks,

Norm

2

I dont think Matillion has this feature. The best possible way is to list all the jobs in a folder using an API call and then loop through the list of Orchestration jobs.

 

A simple solution for this one is to create an Orchestration of Orchestration jobs and then trigger that one orchestration job , which in turn calls the other orchestration jobs. This way, you will also be able to manage the dependencies between the Orchestration jobs.

 

 

3

The API approach looks promising.

There may be hundreds of jobs with no real dependencies on one another so having a large Orchestration or Orchestration jobs might get difficult to manage.

Thanks for the idea!