Have you used Matillion in your Data Science initiatives?

Matillion ETL
1

I'm curious if there's anybody here uses Matillion for Data Science? Maybe we can share some best practices here as well. Thanks!

2

Much of our ML and data science initiatives are driven using derivative files, that is to say, files where some preparation has been done against the database and the results have been stored as files or external tables. To that end we make extensive use of Matillion to perform the ETL (or ELT) required to take our data sources and prepare them for use in various models.

3

Thanks @EdwardHunter​ for sharing! Have you used Matillion in putting the results as files?

4

Yes, absolutely. We will frequently ingest data from a variety of sources, perform transformations and aggregations, categorizations etc., and output the results as files for use by our data scientists or other consumers.

5

I see. Seems like Data Scientists nowadays are more comfortable working on files instead of a database?