When using the JDBC Incremental Load Component, what happens when an incremental key column is specified, but no unique key column is provided?

Matillion ETL
1

We have a number of projects all using the Incremental Load component, and some have very complicated source tables.

We have had mixed results with not explicitly giving a key column, sometimes Matillion seems to 'find' a key to use and actually incrementally load without issues, and sometimes we end up with duplicates in our target tables.

 

Some of these source tables have very large composite keys (10+ fields), so if there was a way to not have to explicitly define these it would save a lot of time. However, we need some confidence in Matillion's process before I can fully recommend our team skips the unique key all together.

2

Hello,

Thank you for your post, we have some documentation on JDBC Incremental Load here this should help, If you are still stuck, please do reach out to our support team via support@matillion.com they will be able to take a more in-depth look at this for you.

Please do pop back and let us know the solution to this :)

Kind regards, Joe

3

Joe,

 

I've read through the link you provided, as well as the Help tab in Matillion. It appears both sources of information tell us where/how to define the primary key but no further details than that. We are wondering what happens when a PK is not specified, and what Matillion does in that scenario.

 

I'll put in a support ticket and see if they can provide any further information.

 

Thanks,

 

David

 

 

 

4

Thank you for your reply.

Our support team will be able to look into that for you, please pop back here and let us know the solution.

Kind regards, Joe