At this stage we are developing an Operational Data Store (ODS) where we will source data from Aurora/RDS PostgreSQL and collect all production data into Aurora/RDS PostgreSQL. Afterwards, the next leg of our operation is to build a Redshift Cluster DWH (star schema) to be sourced from our ODS and potentially other Data Sources.
Want to know if there is a means to pull data from RDS and upsert into RDS at this time.
Hi there @mmaayeh, are you still looking for an answer to your question? Have you had a chance to check out our Documentation and support sites? I had a quick look, let me know if these help at all. RDS Query or How To: Connect RDS Aurora PostgreSQL via RDS Query Component
Let me know if those help at all,
Claire
Hi Claire,
Sorry for the late reply. I was out of the office for a long period.
I have found a work around for this issue. The idea was that the RDS Bulk Copy has bugs that prevented the usage of this component. The typical path of Matillion ETL is from Data Source to Redshift Cluster then to Data Destination (PostgreSQL Aurora/RDS). The means to do this is via the RDS Bulk Copy however, that required a workaround using python script to avoid using this component. A case has been opened with Matillion regarding this case and awaiting resolution.
thank you,
Muhanned