Hello,
We have a Matillion data pull job that uses a grid iterator to pull data from salesforce using the Salesforce Query component. The Grid Iterator is running in concurrent mode.
We are noticing the that once the Staging in S3 is complete, Matillion process is waiting a long time for the Snowflake to copy the data from S3 to Snowflake table.
Could this be because Snowlfake is slow in copying the data from S3? If so, how can we optimize it?
Should we increase the Cluster Size or should we increase the Number of Clusters?