Is MDL Batch Load incompatible with MSSQL datetime2 date type?

Data Productivity Cloud
1

I have a Batch Load job that is loading data from a MS SQL Server source into Snowflake.

 

I'm trying to use a "lastUpdated" field as a high water mark for the sync.

 

On most source tables, this field is specified as type "datetime2(7)". When MDL detects the source field however, it detects this as "varchar".

 

Interestingly, on checking another source table where the field type is "datetime", MDL detects this as "timestamp".

 

Is there a compatibility issue with MDL and SQL Server datetime2 type?

 

If so, will MDL detecting this as varchar cause issues with using this field as a high water mark due to character sorting potentially behaving differently to native dates?

2

Hello @gshenanigan​

Thank you for your post, unfortunately, that is not currently something that is possible within MDL although, I would recommend raising an idea for this, have you had a look at the ideas Portal? the idea may have already been raised, if not raise it and our product team will review that for you.

Kind regards, Joe

3

Hi ,

​

I​ am using Microsoft SQL server output component to load snowflake data to sql server, I have datetime field last_updated_ts which is tinestamp_ntz(9) in snowflake and datetime2 in sql server and it is failing with the error scale 9 is invalid. Looks like matillion does not support. Any suggestions?

​

Thanks