Hi all! It's Ian from Matillion's Customer Success team, with a updated snapshot of the most popular Community tags in the last six weeks.
Matillion ETL itself, “VM”, “Jobs”, “SQL”, “Run” and “Scripts” all tie in with a new community sourced JDBC driver for Matillion ETL itself which we have just added to the Matillion Exchange.
It’s a standard JDBC type 4 driver, which you can install by following this guide.
The JSON fragment that you’ll need for /usr/share/emerald/WEB-INF/classes/jdbc-providers.properties looks like this
{
“name”: “METL”,
“driver”: “com.thereisnogravity.Driver”,
“url”: “jdbc:metl://”,
“fetchSize”: “500”,
“limit”: “limit-inline”,
“allowUpload”: true
}
It’s also available for download on the Exchange.
Once installed
- Create an ordinary Database Query component using the driver
- Provide the username and password of a user that’s privileged to use the Matillion API
You will find various built-in data sources available, including
- group, project, version and job - standard Matillion metadata
- schedule and environment - covering all projects that the user is permitted to access
- runningjob - showing a snapshot of all running jobs in all projects
- joblaunchstats - gives you insight into any latency issues which are preventing your jobs from starting running immediately after they are launched (for example trying to run too many jobs at once, or trying to run the same named job twice in parallel). The statistics will cycle once per week, so you should aim to capture them to a more permanent table
- instance - giving you basic metrics on your VM, such as CPU, network, timezone etc
The driver will work better on more recent versions of Matillion, and I’d recommend sticking to Basic mode rather than using the SQL editor.
You’re free to use this however it’s useful, but one suggested Orchestration Job structure would be like this:
More posts to follow, but meanwhile your comments, feedback and feature requests are welcome as always.
Best regards,
Ian