I want to make the job documentation more easily available to my analysts. I don't want to have to remember with each release to go and regenerate the job documentations, plus the generation can be done at different levels either at the top orchestration that is scheduled or one of the sub components that handle a single table.
What I was thinking of doing if there was some way to mimic the folder structure in Matillion as a website site and analysts could see the HTML documentation that is generated for each job. This would provide more self service for analysts who are curious about and transformations that are happening in the jobs.
Hey Michael did you come up with a way to do this?
I was looking to see if Matillion had a way to generate the Job documentation HTML file through the API similar to the reasons you stated above.
What I've learned is the generation of the documentation can't be done through an API call. At least not without a lot of custom code. The generation of the documentation relies on the web browser (I suspect as a client side java script engine) to create the docs. To do this automatically this is the setup I'm guessing would work but haven't tried it:
Use a technology like Selenium that would run a full browser on a VM, automate the clicks through the application, download the documentation, and then from move the downloaded files where ever you need them. I haven't tried this and I'm sure there's lots of complications need to try and get this to work.
Since making my post, Matillion has changed to their Hub model which makes it possible for more view only access to the application which for now kind of helps with the transparency and documentation of processes.