Is there a way to pass grid variables to a transformation job and have the "Read"/"Write" components access the variables?

Matillion ETL
1

I'm working on CDC for my warehouse, and with the warehouse size increasing, I want to add a scalable methodology for my CDC. The CDC is done in 3 steps, 1. Insert New Records 2. Insert Changed Records 3. Close out Changed (Old) Records.

 

Hoping to complete the 3 steps through variables for table names so I don't end up with a hundred components doing the same thing.

 

I know one way to go about it is Shared Jobs, but just looking for some direction.

2

You can pass Grid variable values to transformation using Grid Iterator. You need to configure the grid values for each iteration to pass in transformation using environment variable.

 

Check this --> https://documentation.matillion.com/docs/2943274