Hi All, Based on your experience, i would like to know the typical time it usually takes to migrate the Matillion ETL jobs to DPC. i know this depends on lots of factor. do we have any calculator or estimation sheet template that i can refer

Migration
, ,
1

Hi All,

Based on your experience, i would like to know the typical time it usually takes to migrate the Matillion ETL jobs to DPC. i know this depends on lots of factor. do we have any calculator or estimation sheet template that i can refer.

2

Hi @rahulsahay123​

Small Migration (a few simple jobs): Typically, this can take a few days to a couple of weeks, depending on the complexity of the jobs.

Medium Migration (several moderate jobs with some complexity): This can take 2–4 weeks.

Large Migration (many complex jobs or integrations): This can take a couple of months, depending on the number of jobs and the amount of testing required.

3

there is no specific tool dedicated to estimating the time for migrating Matillion ETL jobs to Google Cloud's DPC, but there are ways to estimate the time using:

  1. Google Cloud tools (for infrastructure and cost estimations),
  2. Matillion's resources (for understanding how to migrate ETL workflows),
  3. Custom estimation sheets (based on complexity, number of jobs, and data volume),
  4. Project management tools (like Jira or Asana) to track progress.
4

Hi Rahul,

 

I'm a Product Manager here at Matillion and one of the areas I look after is the DPC Migration Tool. There's no easy answer to the question how long a migration will take as it relies on a number of factors.

 

One key one is making sure you leave enough time to test pipelines after migration, while we aim for compatibility there are some changes behind the scenes that make it worth while to check the migration pipelines work the same.

 

If you want an overview of migration please see this video: https://www.youtube.com/watch?v=9YG-rcrgooI

 

In terms of timelines, it really depends on the complexity

 

1) Project mostly consisting of Transform jobs will be lower effort and need less refactoring, you could probably assume 2 days per pipeline and then some testing

 

2) More orchestration heavy pipelines are likely to need more refactoring, so consider 3 to 4 days here. But obviously this will vary based on complexity

 

3) Some areas in the DPC work differently so refactoring will be necessary, pipelines that involve:

 

  • Python or Bash
  • API Query or API Extract
  • Shared Jobs
  • Automatic Variables

 

are all likely to need more refactoring and thus testing. I wouldn't want to give a time for these.

 

I hope this helps,

Jamie