How to specify a filter parameter when using a custom RSD schema for MongoDB

Matillion ETL
1

I have a mongoDB source that has a lot of sparsely populated fields. The automatic schema discovery, even with a large row scan depth, doesn't always find all the possibilities each time it executes which then I'll get a job failure. This is leading me to use a manually specified RSD schema file.

 

I have the file working and Matillion is finding and loading the data for the columns. What I can't get to work is the ability to use a filter to allow me to incrementally load. There is a date column in the data that I want to be able to pass in a variable value into. With automatic schema discovery this is working, just not with the manual schema file, so there must be a syntax I'm missing in my schema file is what I'm thinking. The error I'm getting is a generic "This feature has not yet been implemented."

 

Here are some of the links I've been following but I'm still stuck

2

I have additional information I thought I'd share here in case someone else stumbles upon this post. I was doing everything right in the RSD defintion and you don't need to do anything special in the RSD to specify which fields can be used to filter on. Discovered the actual issue is with the CData driver that Matillion has integrated. The driver version does not have the ability to use inequalities (<, >, >=, <=). Good news is the latest CData driver now supports this and I'm waiting for a Matillion software release that will distribute this updated driver.