Extract API task history to monitor it

Hi mw,

 

I don't think Matillion implemented paging in that part of the API.. so during processing it tries to hold up to an entire year's worth of task history in memory. That's what can cause the OutOfMemoryError: Java heap errors.

 

According to the docs you can filter by date range... http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/task/filter/by/start/range/date/<startDate>/time/<startTime>/to/date/<endDate>/time/<endTime>

 

SO you may be able to loop for example month by month and fetch a smaller amount of data at one time.

 

If you want quick summary stats you could take a look at Matillion;s own JDBC driver.

 

CK