Hi,
Our Matillion is hosted in GCP VM, and it loads many files to BigQuerry.
I found out that every component using BigQuery takes at least 2 seconds to run. Same results both in DEV and PROD environments.
For example, if I create a dummy orchestration job that runs 8 "select CURRENT_DATETIME() AS Now" in sequence, it runs for 22 seconds. More than 2 seconds to perform a single select, which is really too slow in our context.
A dummy orchestration job that runs 8 bash scripts (echo “Hello world”) runs for 1 second, so I understand the problem is not directly on Matillion VM.
Any idea why we face this slowness on querying BQ from Matillion ? Maybe did I miss something obvious here ?
Thanks for your help,