Generate a dynamic table using Grid Variables

Matillion ETL
1

Prerequisites:

This example assumes that you have created a Grid Variable and are populating it with the column name and data type of the table you are trying to create. This allows the create table to be dynamic depending on how you choose to populate the columns in the Grid Variable.

Steps:

  1. Add a Create Table component after the step populating your Grid Variable.
  2. Configure the Create Table component, when selecting Columns, select Use Grid Variable in the bottom left of the columns configuration window.
  3. Select the column name and type from the Grid Variable in their corresponding option.

    0694G00000LVJpzQAH_0D54G00008d70yQSAQ
    0694G00000LVJpzQAH_0D54G00008d70yQSAQ1734×792 119 KB

  4. Add a S3 Load Component after the Create Table.
  5. Configure the S3 component
    1. In the design pattern, use an acceptable pattern to search for the files. You can use THIS documentation for samples.
    2. You can use variables for Target Tables
    3. Select the proper File Type that you want to un-nest, like PARQUET, JSON, CSV, etc.