Our team is new to Matillion - We have a requirement to decide to do SCD1 or SCD2 based on the columns that have changed

Matillion ETL
1

 

For e.g. we have table1 with 10 columns col1, col2, col3 ... col10. 

If ONLY col1, col2, col3 change and nothing else.. then we do SCD1 else we do SCD2. 

 

Is this possible using detect changes component ?

2

Are you saying for some columns (col1, col2, col3), you only want to store the "current value" (type 1) ... while all other columns require history tracking, so they trigger the creation of a new row (type 2)?

 

If so, then I'm sure you could do that in Matillion. And I do expect that a Detect Changes component would be involved. But there is more to it. I believe I've seen a video in Matillion's YouTube channel which explains how to maintain a type 2 SCD ... and you could modify it a bit to include some type 1 columns.