Hey I am wondering if anyone is using the python component to call a REST API to get data and somehow save the response to Snowflake or Blob storage?
I am running python scripts outside of Matillion to do this because I thought Matillion didn't recommend this and I don't know how to securely handle secrets within the python component in Matillion.
Hi Grant, I completed this project yesterday with our azure admin. This might help:
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python?tabs=managed-identity%2Croles-azure-portal%2Csign-in-azure-cli#upload-blobs-to-a-container
In the python script you need to authenticate and authorize the Azure blob storage.
Next up you need to (we did it with download urls we received from requests) write those url paths towards your azure container to get the csv files inside the azure container.
https://documentation.matillion.com/docs/2937412
Yeah, thanks that is what I do with my python scripts outside of Matillion. How are you securely storing the authentication details?
Gotcha! Makes sense :)
Using keyvault