All API components failed on daily run and was fine the next day

Matillion ETL
1

We had an strange issue on Saturday.

All the API components failed with the error:

(Same error for different APIs on different schedules)

 

Parameter Validation Failure:

Data Source - Could not retrieve a data source.

Data Selection - A valid data source is required.

 

The next run on Sunday was fine again.

No changes before or after.

 

Looking at the catalina logs I found this messages:

 

com.matillion.bi.emerald.component.implementation.orchestration.PropertiesLoader.runSchemasLookup Error running schema lookup

 

com.matillion.bi.emerald.shared.component.implementation.ComponentParamImplementation.checkParameterStates Parameter did not validate successfully. Slot: 6. Name: Data Source. Message: Could not retrieve a data source.. State: INPUT_INVALID

 

 com.matillion.bi.emerald.server.model.util.jdbc.JDBCQueryHelper.getQueryMetadataBasic Retrieving basic mode column metadata for table 

 

com.matillion.bi.emerald.component.implementation.orchestration.PropertiesLoader.getAvailableQueryColumnNames Could not determine result columns for query

 

 

We are on version 1.53.9 on Snowflake.

 

Had anyone an similar failure in the past?

This is the first issue we encountered and some of them are running for over a year now..

Even one job that did 4 retries failed.

 

2

I just want to give an update.

This issue appeared again today.

So this time I was able to check it myself. I could fix this issue by restarting the server.

I opened a ticket now.

I attached a screenshot how it looks like in the ui.

The profile can be selected but the data source selection is not working.

I will keep you updated if we find the root cause of this issue.

 

3

Hey there - we just experienced the same issue last night.

Our Matillion server "wakes up" each night (running on AWS EC2 instance) to run some updates and last night all our jobs scheduled at 2:30 am failed with the same error message.

It was in various jobs and data sources, no consistency.

I'm suspecting there was an internal issue with Matillion to cause this.

We are on Version: 1.61.6 (build a392342-2471)

@MatillionProductTeam​ Any thoughts on this? I can provide logs if necessary...

4

Thanks for keeping the community updated on this and please do let us know what the support team advises and in the meantime if you need anything else, please do not hesitate to ask.

 

Kind regards, Joe