We are making Rest API calls using Python component (Jython) to post some data from Snowflake to a target system. We are sending the data in JSON format. It is working without any issue if the payload size is relatively small but getting 504 timeout error consistently if the payload size increases. We are able to successfully post the request with same payload using Postman from local machine. Moreover it seems the request is not reaching to the target system.
Has anyone faced this issue? Can someone throw some light on the possible areas we need to look into?
Unfortunately, 504 errors when dealing API's can be quite the challenge to track down unless you turn on logging through the various object that are being touched. In a way your Postman example is unhelpful in this situation. I say this because you are likely running Postman from your computer where Matillion is executing within AWS, Azure, GCP, etc. The routing and network equipment your local machine is going through is probably completely different than what the Matillion instance is navigating. Timeouts can show up with there is timeout thresholds set on routers, gateways, etc. Not only in your cloud infrastructure but also on the API hosting side if they are not in the same cloud provider account. You might start by looking at the logging for your network infrastructure within the cloud provider you are running in. There's a good chance that with verbose logging on all services, you will find something that will lead to the culprit.
The only other thing I can think of is setting the Python request timeout higher but I don't believe that will help since the 504 is typically generated from server side that is hosting the API not the client side.
Thank you very much for your reply. Was trying all possible options to troubleshoot the issue. Here is the latest update on the issue. Thought of sharing with you if you could provide some insight on this. Please note that we are using Jython and we are not getting this error always but only when the payload size is little big.
Checked with network team. Apparently no issues found.
Tried posting the same payload from Matillion VM using curl and it was successful.
Tried posting the same payload using Matillion Python 3 (used requests) module and it was successful.
Seems like we are facing this issue only in Jython (Used urlib2 for making the API call).
Any idea if we need to change any settings in the tomcat server configuration or anywhere else.
I like the troubleshooting you did. This is extremely helpful and definitely points to either Jython or the urlib2 module. This is not really surprising. Although I have not ran into this particular issue with Jython, I have ran into others.
My usual recommendation to anyone that is using or looking to use Jython is to only use it as an absolute last resort. As of right now, there is no Jython 3 which means it's extremely behind functionality, security best practices, and is very hard to find help with it anymore. There is also a good chance that it could be deprecated at some point.
With that said, Jython is wrapper around Python 2.7. That comes with some limitations and caveats right from the start. The first is if you use imports like the "requests" module, those modules will (have to) be designed for Python 2.x which is being deprecated. Depending on the API you are trying to hit urlib2 may be the culprit. That module was design for the most basic use cases around API's. It doesn't support a lot of the more modern API features. You may be able to set the timeout on the socket which is lower level than urllib2 but urlibe2 uses sockets for it's transport layer. Take a look at this: https://docs.python.org/2.7/howto/urllib2.html#sockets-and-layers
My advice would be to ditch Jython unless it's absolutely necessary. For pulling data from API's I would try to leverage API Profiles as much as possible. If that's not possible then falling back to Python 3 or bash is ok.
I hope this helps in some way. Post back if you still have issues or need more direction. Thank you!