# Issue with flattening and extracting JSON data using flatten variant

**URL:** <https://matillioncommunity.discourse.group/t/issue-with-flattening-and-extracting-json-data-using-flatten-variant/2328>\
**Category:** Matillion ETL\
**Created:** [November 1, 2023, 2:24pm UTC](https://matillioncommunity.discourse.group/t/issue-with-flattening-and-extracting-json-data-using-flatten-variant/2328 "2023-11-01T14:24:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![DBATT](https://avatars.discourse-cdn.com/v4/letter/d/34f0e0/32.png) [@DBATT](https://matillioncommunity.discourse.group/u/DBATT)\
**Post date:** [November 1, 2023, 2:24pm UTC](https://matillioncommunity.discourse.group/t/issue-with-flattening-and-extracting-json-data-using-flatten-variant/2328/1 "2023-11-01T14:24:25Z")

</div>

I am having issues with trying to extract values from a JSON Array and utilizing the flatten variant component. Below is the JSON format:

​

[

&nbsp; {

&nbsp; &nbsp; "key": "ga\_session\_id",

&nbsp; &nbsp; "value": {

&nbsp; &nbsp; &nbsp; "double\_value": null,

&nbsp; &nbsp; &nbsp; "float\_value": null,

&nbsp; &nbsp; &nbsp; "int\_value": 1698415322,

&nbsp; &nbsp; &nbsp; "string\_value": null

&nbsp; &nbsp; }

&nbsp; },

&nbsp; {

&nbsp; &nbsp; "key": "firebase\_event\_origin",

&nbsp; &nbsp; "value": {

&nbsp; &nbsp; &nbsp; "double\_value": null,

&nbsp; &nbsp; &nbsp; "float\_value": null,

&nbsp; &nbsp; &nbsp; "int\_value": null,

&nbsp; &nbsp; &nbsp; "string\_value": "auto"

&nbsp; &nbsp; }

&nbsp; },

&nbsp; {

&nbsp; &nbsp; "key": "engagement\_time\_msec",

&nbsp; &nbsp; "value": {

&nbsp; &nbsp; &nbsp; "double\_value": null,

&nbsp; &nbsp; &nbsp; "float\_value": null,

&nbsp; &nbsp; &nbsp; "int\_value": 1318,

&nbsp; &nbsp; &nbsp; "string\_value": null

&nbsp; &nbsp; }

&nbsp; },

&nbsp; {

&nbsp; &nbsp; "key": "ga\_session\_number",

&nbsp; &nbsp; "value": {

&nbsp; &nbsp; &nbsp; "double\_value": null,

&nbsp; &nbsp; &nbsp; "float\_value": null,

&nbsp; &nbsp; &nbsp; "int\_value": 1,

&nbsp; &nbsp; &nbsp; "string\_value": null

&nbsp; &nbsp; }

&nbsp; }

]

​

My desired output would be a new column for each object in the array above:

​

ga\_session\_id | firebase\_event\_origin | engagement\_time\_msec | ga\_session\_number

---------------------------------------------------------------------------------------------------------------------------------------------

1698415322 | auto | 1318 | 1

​

​

Here is how the "flatten variant" component is setup:

​

​

​

 ![0694G00000UuHaUQAV_0D54G00009irMUUSA2](https://us1.discourse-cdn.com/flex002/uploads/matillioncommunity/original/1X/5db3307f554d77e5ba78052f4822a4ec9c654892.png)

 ![0694G00000UuHaeQAF_0D54G00009irMUUSA2](https://us1.discourse-cdn.com/flex002/uploads/matillioncommunity/original/1X/b33f9b977c4e0b09865eba0756db7597fce44e27.png)

​

I am able to get the four rows flattened but any time i try to extract the “key” value or value within, it get NULLs.

​

I’ve also tried the “extract nested data” component which identifies the “event\_params” as a json structure but nothing occurs when I select autofill.

 ![0694G00000UuHcQQAV_0D54G00009irMUUSA2](https://us1.discourse-cdn.com/flex002/uploads/matillioncommunity/original/1X/c87081e69d6d55912121a1c2ce28fa15f77145fe.png)

​

Any help/tips would be appreciated!

---

<div class="post-metadata">

**Author:** ![MichaelBoucht](https://avatars.discourse-cdn.com/v4/letter/m/3da27b/32.png) [@MichaelBoucht](https://matillioncommunity.discourse.group/u/MichaelBoucht)\
**Post date:** [November 13, 2023, 6:49am UTC](https://matillioncommunity.discourse.group/t/issue-with-flattening-and-extracting-json-data-using-flatten-variant/2328/2 "2023-11-13T06:49:44Z")

</div>

Hello,

if using Flatten Variant, you can get your JSON pulled into separate columns first,

soemthing like this:

 ![069Tm000000Czm5IAC_0D7Tm00000024txKAA](https://us1.discourse-cdn.com/flex002/uploads/matillioncommunity/original/1X/1880b4131e34a80d3fe32e5efe7bd424df5c519b.png)

&nbsp;

but this will have 4 different rows from your data, use case-when or coalesce, or similiar, to identify the correct value per name.

which you would then need to transpose to your desired output format.

&nbsp;

Or if the dataset is ‘static’,

you can always dig it out, using Snowflake SQL on a Calculator component, like this:

If it’s always the same order of 4 keys,

“event\_params”[0]:value:int\_value as ga\_session\_id

&nbsp;

or if it’s not guaranteed the same order, but always 4,

&nbsp;

CASE

WHEN “event\_params”[0]🔑:VARCHAR = ‘ga\_session\_id’ THEN “event\_params”[0]:value:int\_value

WHEN “event\_params”[1]🔑:VARCHAR = ‘ga\_session\_id’ THEN “event\_params”[1]:value:int\_value

WHEN “event\_params”[2]🔑:VARCHAR = ‘ga\_session\_id’ THEN “event\_params”[2]:value:int\_value

WHEN “event\_params”[3]🔑:VARCHAR = ‘ga\_session\_id’ THEN “event\_params”[3]:value:int\_value

&nbsp;&nbsp;ELSE NULL

END as ga\_session\_id

&nbsp;

I would never do unless 100% sure it’s always like that. to keep generic and easily add more, I would use flatten variant as in example one, and then just transpose, then you can semi-automate the column genereration, or fully automate it using grid variables, just keep all the values as varchar until they are actually used somewhere.

&nbsp;

Hope this makes sense and helps you forward 🙂

&nbsp;

-Michael

---

<div class="post-metadata">

**Author:** ![DBATT](https://avatars.discourse-cdn.com/v4/letter/d/34f0e0/32.png) [@DBATT](https://matillioncommunity.discourse.group/u/DBATT)\
**Post date:** [December 5, 2023, 9:19pm UTC](https://matillioncommunity.discourse.group/t/issue-with-flattening-and-extracting-json-data-using-flatten-variant/2328/3 "2023-12-05T21:19:56Z")

</div>

Hey Michael,

&nbsp;

Thanks for the response and explanation! Unfortunately the JSON is dynamic and there is no set structure i can use to call out the indexes of the JSON which is why I was hoping to utilize Matillion for this since the code to write it in snowflake would require multiple stored procs (willing to do if unable to figure out).

&nbsp;

I am still struggling to get the key value and their corresponding values from your screenshot you provided. I always receive back NULL values and just one row (not the four you mentioned). Not sure what I am doing wrong in Matillion but I have ensured that the JSON array columns are converted to VARIANT type before pulling in the flatten component variable.

&nbsp;

Thank you~!
