How to ensure german characters load properly in a Staged file in S3 using the RDS Component

Data Productivity Cloud
1

Using the RDS component to query an Aurora database and load the data into Snowflake. The tables I am querying in Aurora are Latin1. Snowflake requires UTF8 I have trying converting in the mysql query so that the data goes into the staged CSV with the special characters (ü, ß, etc) intact, but nothing seems to work. The characters are always incorrect in the CSV, thus also in Snowflake. Has anyone solved this problem? Are we missing a property in the RDS component that would help with this conversion?

2

Hi @saundrah.humphrey​ and thanks for your post. I've checked in with the team and they are investigating. I will let you know as soon as I know more.

Many thanks,

Claire

3

Hello

 

To ensure German characters load properly in Snowflake, convert the data in your MySQL query using

CONVERT(column_name USING utf8)

 

to ensure UTF-8 encoding. Verify the staged CSV is saved with UTF-8 encoding, and check if your RDS component supports setting

characterEncoding=UTF-8

 

. In Snowflake, define a

FILE_FORMAT

with

ENCODING='UTF8'

when loading the data. Validate the data at each step—query output, staged CSV, and Snowflake load—to ensure characters like ü and ß appear correctly.

 

Thank you !

4

Thanks Claire! I also have been trying with the JDBC component and was having the same issues. I made a support case and got some hints, so will try them and let you know. Having the characterEncoding property available in the list of JDBC Options in the RDS Component properties as it is in the JDBC Component would be a nice addition.