Here's the story! Our use case involves using a rest API (the Tableau API here) to obtain an PNG file, and loading that PNG file into Azure Blob storage for web consumption. So far, I've been able to submit a GET request using the python component in Matillion to obtain the data. The file comes back in the body of the response, which is fairly standard. I'm currently using urllib.request for the request, but open to other options.
My issue is that I don't know how to 'stage' the file in Matillion using the python component, then transfer it from Matillion to Azure. The File Transfer component doesn't have a source type option for something like that. I was hoping there was a 'local' source type, maybe.
The API I'm working with uses a short-lived auth token in the header to grant permission. Otherwise, it just uses query parameters to define the request.
I know that this is basically just SFTP server work, and there are better tools out there. In our restricted permissions environment, I'm hoping I can keep our tech stack simple for today, and get this done using Matillion.
I appreciate any and all advice here. Thanks!