# Export data generated from python to snowflake **URL:** \ **Category:** Matillion ETL\ **Created:** [April 24, 2022, 5:51pm UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733 "2022-04-24T17:51:45Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![yulong.guan](https://avatars.discourse-cdn.com/v4/letter/y/b38774/32.png) [@yulong.guan](https://matillioncommunity.discourse.group/u/yulong.guan)\ **Post date:** [April 24, 2022, 5:51pm UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/1 "2022-04-24T17:51:45Z")
i have created a python script which will send a get request from a website, i can get the data in json format and i would like to use Matillion and place it to snowflake . i have tried to use grid variable and export it to a table, but failed to do so . any suggestion how should i do the process ? thanks ---
**Author:** ![luis.arteaga](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@luis.arteaga](https://matillioncommunity.discourse.group/u/luis.arteaga)\ **Post date:** [April 25, 2022, 7:14am UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/2 "2022-04-25T07:14:43Z")
Where do you execute the python script? On the Matillion VM?   I would recommend that before you extract data using a Python script, you first evaluate the Matillion components such as "Data Transfer", "API Extract" (incl. API profile) if they fit your use case.   If you execute the Python code using the "Python Script" component and select the local storage as the storage path, your JSON file will be stored on the Matillion VM. Here I don't think there is a simple drag and drop component that copies the data from the VM to Snowflake. Sure you will be able to find the file path on the VM, but you would have to connect to Snowflake via the Python script. Before that, you would have to install the Snowflake Python Connector on the VM: https://docs.snowflake.com/en/user-guide/python-connector.html and make sure that the credentials are not stored in the script (keyword: information security).    Matillion is designed to work with a cloud blob storage (Azure, AWS or GCP) and a cloud data warehouse engine (e.g. Snowflake). Therefore, your focus should be on how to get the data into a cloud blob storage.  From there, you could then use a corresponding Cloud Storage component (e.g. Azure Blob Storage).   API Extract component: [API Extract | Matillion ETL Documentation](https://documentation.matillion.com/docs/1959484#:~:text=The%20API%20Extract%20component%20lets,structured%20data%20that%20requires%20flattening.) API Profile: [Manage Query Profiles | Matillion ETL Documentation](https://documentation.matillion.com/docs/2514247) Data Transfer component: [Data Transfer Component (matillion.com)](https://documentation.matillion.com/docs/2953509) Cloud Storage components: [S3 Load Component (matillion.com)](https://documentation.matillion.com/docs/2042349) , [Azure Blob Storage Load (matillion.com)](https://documentation.matillion.com/docs/2954735),   ---
**Author:** ![luis.arteaga](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@luis.arteaga](https://matillioncommunity.discourse.group/u/luis.arteaga)\ **Post date:** [April 25, 2022, 9:36am UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/3 "2022-04-25T09:36:13Z")
Could you specify: "[...] a python script which will send a get request from a website"? I assume you extract data from a website via GET Method? Do you use a REST API? ---
**Author:** ![yulong.guan](https://avatars.discourse-cdn.com/v4/letter/y/b38774/32.png) [@yulong.guan](https://matillioncommunity.discourse.group/u/yulong.guan)\ **Post date:** [April 25, 2022, 9:53am UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/4 "2022-04-25T09:53:13Z")
yes , the script i wrote is just data retrieve from GET request, then i will store it as json/CSV format, ultimately move it to snowflake. ---
**Author:** ![chandrabr.in](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@chandrabr.in](https://matillioncommunity.discourse.group/u/chandrabr.in)\ **Post date:** [April 28, 2022, 1:54pm UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/5 "2022-04-28T13:54:30Z")
Hey , How you have loaded the data into S3bucket from url , Can you please update your code, I have same requirement but unable to do , Please help. ---
**Author:** ![yulong.guan](https://avatars.discourse-cdn.com/v4/letter/y/b38774/32.png) [@yulong.guan](https://matillioncommunity.discourse.group/u/yulong.guan)\ **Post date:** [April 25, 2022, 7:29am UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/6 "2022-04-25T07:29:40Z")
meaning to say i should upload the data query via python to s3 bucket , then i will load the data from s3 to snowflake? ---
**Author:** ![luis.arteaga](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@luis.arteaga](https://matillioncommunity.discourse.group/u/luis.arteaga)\ **Post date:** [April 25, 2022, 12:29pm UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/7 "2022-04-25T12:29:10Z")
If you should request a REST API via GET method, then I would use the Matillion component API Extract instead of the Python script. See https://documentation.matillion.com/docs/1959484   The advantage is that you can store the credentials in the API profile. The sensitive information would then be masked for others and you can conveniently access the credentials via the API Extract component.   If there is no other way and you have to rely on the Python script, you have several options. I see two options for Snowflake and AWS.   1. install Boto3 (AWS SDK) on the Matillion VM. According to the documentation, the Python script would then use your AWS credentials that you used in the Matillion setup. See https://documentation.matillion.com/docs/2234735 2. run the Python script outside Matillion. See https://documentation.matillion.com/docs/2972828   I have not yet been able to implement either option in practice (I work with Azure and Snowflake). Feel free to share your experiences here.   What advantages do you see with the Python script?   ---
**Author:** ![yulong.guan](https://avatars.discourse-cdn.com/v4/letter/y/b38774/32.png) [@yulong.guan](https://matillioncommunity.discourse.group/u/yulong.guan)\ **Post date:** [April 25, 2022, 1:08pm UTC](https://matillioncommunity.discourse.group/t/export-data-generated-from-python-to-snowflake/1733/8 "2022-04-25T13:08:33Z")
thanks for the reply . i do have a lot of variables for the url parameters in a loop and no credentials are needed to do the get request. hence I'm using python script to get the data. I will upload it to a s3 and store the file name as grid variable and load the s3 , extract it to snowflake