Hi - We have a number of use cases that have required using the Python component. Some of this code is getting pretty repetitive (e.g. classes we've written as API clients, interacting with AWS resources, etc.)
Rather than having to copy, pasta and then maintain that code across numerous components, we’re looking for a solution in Matillion to allow us to effectively manage and reuse this code.
Ideally something like being able to `import` our own modules, but with an interface that's less cumbersome than needing to SSH into the box and manually manage in the file system.
Thanks in advance for any advice, tips! Appreciate it.
Not sure if any of your Python scripts can be run separately using parameters, but if they can, you could create a shared job that consists of a just a start component and a python component with the reusable code (https://documentation.matillion.com/docs/2942889#overview). If you need to be able to import the scripts to call from other scripts, this will not work.
As an example, I use a shared job for sending emails in my jobs (see below) then set job variables to be used in the code (bolded variables):
Ah, yeah this is a clever solution that I think will work for some of our use cases where we can parameterize the operation!
But yes, still have some use cases where we’re needing to actually import modules (or something that would resemble being able to pull in and invoke various code within the Python Script component).
