DPC new Streaming Pipeline - private key on KeyVault

Data Productivity Cloud
1

Hi all, sharing this here as reference so next time someone can solve this easily. Not Matillion’s fault, it was just annoying..
Environment: Snowflake using key-pair authentication, Streaming agent and kv on Azure

when creating a Streaming Pipeline on DPC, first you go through the destination variables.
Snowflake 2FA is not working as smooth as I want (asking for 2FA), I opted for key-pair.
I needed to save the private key to authenticate to Snowflake from Matillion side, on Azure key vault.

While creating the secret on Azure, if you copy-paste the PK value, all new lines will be replaced by some other character. Matillion will keep throwing errors. no connection, no pipe.

Instead, use Azure CLI and set/update your private key directly to the vault via the file (so the content stays intact):

az keyvault secret set --name mykeynameonvault --vault-name test-kv --file .\private.key

works like a charm.

Solution reference is here on stackoverflow
Doubt with the key? Check Matillion support article
Matillion’s Snowflake streaming destination docs
and finally - Matillion “Storing the private key” documentation

2

Hello @altinturk

I appreciate you taking the time to share this solution with the rest of the community, its great to hear how you overcome obstacles when using the product.

This being said, I think this is also a valuable piece of feedback for me to pass onto the Matillion Product Team, which I will do right away. I also want to add if you have any ideas or improvements for our product, please do not hesitate to raise them on the Matillion Roadmap.

Kind regards, Joe