I need to read data from AWS Aurora and write to Snowflake, Data Loader is not connecting Aurora/MYSQL DB (Communication link error is coming) appreciate any suggestions in this regard.
Hi @kM690332, I don't know much about the data loader but my mind immediately goes to connectivity with the Data Loader which is outside AWS account and VPC that are hosting Aurora. The Data Loader is trying to reach into the AWS account and one of the first stops is with your VPC and subnets. If those services are blocking internet traffic from coming (which is good security practice) then the Data Loader is not going to be able to make that connection and thus you will receive a communication link error. You will need to get the IP range that the Data Loader is using whitelisted in the VPC setup to make that connection. If you have a network team, they should be able to help with that.
The other possible failure is at Snowflake. Again, the Data Loader needs to connect to Snowflake. I would really hope that you are using Network Policies in Snowflake. If so, you will need to whitelist the Data Loader IP's there as well. I cannot stress this enough. If you are not using Network Policies in Snowflake, you are sitting on a timebomb and data breach. Those are the situations that land companies front and center in the news. It's also a reason why many companies have gone out of business. (i.e. Ransomware strikes again).
I hope this helps!
Thank you Bryan. You are absolutely correct, I tried with public subnet and it is working flawlessly on Aurora Postgres flavor, I tried with same exact setting on MySQL flavor, it is not working, even tried with older versions 5.x no luck. I am not sure why data loader is treating MySQL differently. anyhow working with our Network team to find out more in this regard.
Sounds like you are headed in the right direction. Let us know if we can help more.