Out of Memory - Jobs not releasing memory for certain connectors

Matillion ETL
1

Just in the past week we started experiencing a lot of "out of memory" errors. We hadn't been collecting memory usage until this week but now that we are, we've observed that any data coming through the MongoDB or the RDS doesn't release the memory after the job has completed.

 

Other jobs that are using the Database Query component, they barely even make a blip on the memory usage. Though the data coming through that type of connection is just as big or bigger.

 

Has anyone else experience a similar issue and found a resolution? Any tips what to go looking for? Is there a way to force tomcat to be more aggressive with it's garbage collection?

2

Hi Michael Black,

 

I can see you have this raised with our support team, who are based place to help you resolve this issue, I am sorry you have met this.

But please do keep us posted on how you get on and share your findings with the community.

 

kind regards, Joe

3

Hi Michael,

 

We are also finding similar issues when using a 3rd party driver with the Database Query component. Please do let us know if you find anything useful to help us out with the memory consumption issues that we're facing.

 

Cheers,

Tom

4

Hello,

 

I reached out to our support team with regards to this as I see other users have encountered the same or similar issues.

 

There can be a few things that could cause a memory issue, enabling the heap dump files will really help identify what has caused this. We have some instructions on how to do so here. https://documentation.matillion.com/docs/2959240#enablingfinding-heap-dumps-on-your-instance

 

But to outline a few recurring causes of this, they are as follows:

  • Python Script which happens to be using a lot of memory
  • A database query that fetches a lot of columns/wide columns
  • An iterator within an orchestration job, this happens often when this is in concurrent mode, and with any of the aforementioned causes are contained within.

 

I hope this helps, if you are still experiencing memory issues, please do reach out to the support team, but as always please do share your findings and resolutions here with us in the community.

 

Kind regards, Joe