Hello All,
We are running into an issue with our CDC jobs that is causing data to not be replicated accurately to our destination. Please see the details below.
Source DB: Oracle RDS
Target Database: Snowflake
Source Data:
Target Data:
Note the difference in RQSTR_ST_CD and PRCD_DT between the source and target tables.
Exact record found in the CDC S3 bucket:
Issue Description:
Note the initial insert record flagged with an ‘I’ where most fields are filled in properly.
The next record is flagged for an update with an additional field populated because an additional field is updated in the source. But this time all other fields are now NULL.
So in our target table, after the record is inserted and then updated the only field we see populated is SRC_ID.
Has anyone ever encountered this before? Any suggestions??
TIA!!!
Hi @Adwait_K ,
How are you capturing your CDC records? Are you using AWS DMS/Oracle REDO logs to capture CDC changes on a table/data?
Looking at the CDC S3 file, seems like it is capturing only updated fields in case of UPDATEs.
There might be a setting to capture ALL Fields instead of CHANGED fields only, while exporting the data using AWS DMS.
Hi @AnudeepK ,
Yeah, we are using AWS DMS (RDS oracle - binary reader) to capture changes. I was looking for a setting to change the capture settings but wasn't able to find anything in DMS on the source endpoint side. Do you think this setting might be available anywhere else within the DMS task?
Thanks
Hi @Adwait_K ,
Correction from my earlier comment. The issue is with Oracle Redo Logging.
You can check with your Oracle DBA to enable Supplemental Logging on the database. This will help to capture all the columns from its previous row along with the updated columns.
For more details, please refer this link.
Supplemental Logging (oracle.com)
