Managing change in the column definition from source DB

Matillion ETL
1

Hello! 

 

We have a custom B2C digital platform with MongoDB as the backend. Recently, after its integration with one of our internal systems, a few tables in the platform's database backend were impacted.

In one of the tables, a column, initially configured as type NUMBER(10), now also has character elements. The MongoDB load component recognises this column as Number(10) and defines it accordingly on Snowflake before loading the data. But, during the copy process, Snowflake throws an error because of the mismatch.

```

Numeric value 'XX123456' is not recognized

 File 'ac9274f7-03cb-4370-8220-eec8a11deb65/ac9274f7-03cb-4370-8220-eec8a11deb65_6', line 1, character 63

 Row 1, column "TABLE_NAME"["columnName":11]

```

 

Posting it here for help and suggestions on how this can be overcome.

 

Thanks,

Prajwal

2

Have you tried using convert type component after mongo db read to make it varchar?