We are trying to explore Github actions for migrating the code between environments for Matillion. Our Dev, Test and Prod Matillion instances are mutually exclusive. We have been thinking of using one Git repo to be used with Long Lasting Branches for each evironment (viz, Dev_Master, Test_Master, Prod_Master/Master) .
Developers create feature branches , commit the changes to feature branch and merge it into Dev Master. We are planning to create an automated pull request using Github actions to merge the branch into Test Master (following the approval process) . And similarly to the Prod Master when release happens.
Has anyone used this approach to migrate/deploy their code between environments?
I have not used GitHub for the purposes of Matillion deployments but we do use it for dbt deployments/releases and it works quite well for that. I would be interested in your experience with implementing this approach for Matillion. Do you mind posting back if/when you implement this? This may be an approach we take.
Sure Bryan. We are able to achieve the automated deployment of code between the long lasting branches, working on embedding the Approval process into automated Github pull requests. Once we get that sorted, I believe we will be in a good shape. Happy to share my learnings as part of this journey once we implement this solution.
We are also in the same boat trying to automate the deployment using Github actions. Can you please let us know if anyone has implemented automation of Matillion deployment .
It will be great help, if anyone can shed a light on this approach.
We have recently started investigating our development lifecycle more and it quickly became clear that the merging of branches is useless in GitHub if you would like someone else to do a code review. Each change that is committed is nothing more than a bunch of JSON. It would be next to impossible for the reviewer to look through the JSON and understand what changes were made. I can see the use for approvals but even at that you are making the wild assumption that whatever description the creator put in the commit encompasses all the changes he made. As the approver, I would want to know the detail behind the changes. If the changes are just being approved to satisfy some requirement like a SOX process then it makes sense because that's just a formality where you prove that each change was approved by someone.
In my opinion the Git integration in Matillion is not an end to end solution but rather a feature that sales can to say that we have Git integration. Again, in my opinion the Git feature is something most new customers want and then are extremely disappointed once they start using it and create a development lifecycle around it. I am holding out hope that Matillion will make this right. 😀
Hi @AnudeepK, sorry for the extremely late post. I missed your reply for some reason. We are not using the Dbt Core implementation that Matillion has adopted. We are using Dbt Cloud. So that approach is quite different.
Right now we are setting up our repos with Dev, Test, and Prod branches. We do our dev work in Dev or on a feature branch off Dev, then manually merge the changes to Test within GitHub. We have GitHub's branch protection turned on for the Prod branch which requires a pull request from Test to Prod and 1 or more reviewers to approve the change before it's able to be merged. Hopefully that explains our Dbt Cloud dev process.