SAP ECC data replication to sql server

Matillion ETL
1

Hello folks, My client is looking to replicate their SAP ECC (sitting on Oracle) and some SAP BW tables to SQL server. All these are On-Prem and I'm wondering if Matillion could do this. If yes how?

Client has both Matillion ETL and DPC

 

Please provide your suggestions. Thanks in advance for your inputs

2

Hello @nivedhitha.k​

Thank you for your post, you can do this by leveraging Matillion's Extract Load Transform and Sync capabilities.

This first part of the pipeline involves replicating the SAP BW tables to the Cloud Data Warehouse e.g. Snowflake. Matillion has various methods for connecting to these tables e.g. Database Query for Oracle, SAP OData, SAP Netweaver connectors.

Matillion's transformations will then ensure that the data is replicated correctly and perform any necessary cleansing and standardisation.

Finally, Matillion's SQL Server Output connector will be used to Sync the data from the CDW to SQL Server.

The SQL Server can be onprem provided there is a network path from the cloud where Matillion lives to onprem. The CDW will not be on-prem.

I hope that helps, if you need anything else, please let me know, I have hyperlinked some documentation to support this also.

Kind regards, Joe

3

@JoeCommunityManager​ Thank you for taking the time to respond

A few follow up questions

  1. Is it possible to bi-pass CDW and move data between the two on-prem databases directly? Why is a CDW needed?
  2. What are the pre-requisites for connecting to an on prem source like setting up VPN etc?
  3. Can cluster tables be migrated without issues? And what is the best connector for the same?
  4. Can the data loader be used in this case? We do not have any transformations. Looking to just move the data from system a to system b

Thanks,

Nivedhitha

4

Thank you @JoeCommunityManager​ . That was very detailed and to the point.

Appreciate your inputs

and no I do not know the name of the cluster tables

5

Thanks for your reply, I will answer in the same format as you asked so it correlates :)

 

  1.  Not possible to bypass CDW. Matillion is an ELT so all its transformations are pushed down to a CDW.  For this reason, Matillion's EL connectors always target a CDW. 
  2.  You will need a network path between your cloud VPC/VNet and the on-prem source. This can by integrating your private cloud network with your private on-prem network to use private IPs or by opening firewalls to allow specific public IPs.
  3.  Cluster tables are tricky, do you know the name of the table?  You might be able to use the SAP ODP connector for this. 
  4.  Data loader only targets CDW

 

 

Thanks Joe :)