Matillion not escaping all New Lines in detailed_error variable

Matillion ETL
1

We are starting using the Manage Error Reporting feature in Matillion. We are sending a JSON with common Automatic Variables:

 

{

"Notification_Type": "Error",

"Run_History_ID": "${run_history_id}",

"component_id": "${component_id}",

"component_name": "${component_name}",

"component_message": "${component_message}",

"job_id": "${job_id}",

"job_name": "${job_name}",

"project_group_id": "${project_group_id}",

"project_group_name": "${project_group_name}",

"project_id": "${project_id}",

"project_name": "${project_name}",

"detailed_error": "${detailed_error}"

}

 

Matillion is escaping double quotes and New Lines in the variable "detailed_error" pretty well but it is not escaping one New Line character that seems to be inserted between each error message:

 

0694G00000KqsZGQAZ_0D54G00008RKtNrSAL
0694G00000KqsZGQAZ_0D54G00008RKtNrSAL1194×313 56.5 KB

 

We are getting the Payload to Splunk. This one is saying that the JSON message is not correctly formatted (only when there are more than one component failing).

 

Are we doing something wrong? Is there any way to correct this?

 

Thank you in advance,

2

Hi Fernando,

 

I don't have an answer for you, but I was about to post a question to the community on what the best way to log to Splunk would be and it looks like you've found a way. Are you sending to a Splunk HTTP Collector?

 

Also, had you entertained using the Splunk add-on for Tomcat to route all log messages to Splunk?

 

Thanks!

 

Carl

3

Hi @CarlDEPT​ ,

I am using the "Webhook Post" component in Matillion to send a Payload to Splunk.

I am not 100% sure what we use in Splunk because that is not my part in the project but yes, it looks like some kind of HTTP Collector. The URL looks like this:

https://splunk.myclient.com:8088/services/collector/raw?token=XXXXX-XXXXX-XXXXXX-XXXXXX-XXXXXXX

Regarding Tomcat, I don't think the Splunk team is monitoring Matillion with this add-on but again, is not really my area. I am working for a big client therefore there is a fair amount of people involved taking care of various parts.

All the best!