S3 Unload appears to produce invalid JSON?

I need to produce json files from a fairly simple table. Using settings shown below, the file being produced isn't correct JSON syntax. Specifically, we're missing commas between the arrays, and the [ ] brackets at the start and end. The partner with whom I'm trying to integrate can't accept this invalid json, so I am stuck.

 

 

Here’s what is being produced:

 

{

“BRAND”: “CoMotion”,

“COLOR”: “Premium Paint Color”,

“DESCRIPTION”: “S/O CoMotion 50cm Ochoco 18 Speed Gear Box and Pathfinding Lighting Package”,

“EXPERIENCE”: “TOURING”,

“ID”: 210000124192,

“NAME”: “S/O CoMotion 50cm Ochoco 18 Speed Gear Box and Pathfinding Lighting Package”,

“PRICE”: 5939.99,

“PRICEWPROMO”: 5939.99,

“SIZE”: “Painted Silca Pump”,

“URL”: " "

} {

“BRAND”: “Orbea”,

“COLOR”: “Blk/Red”,

“DESCRIPTION”: “CR Orca OMR”,

“EXPERIENCE”: “COMPETITIVE ROAD”,

“ID”: 210000124221,

“NAME”: “CR Orca OMR”,

“PRICE”: 1900,

“PRICEWPROMO”: 1900,

“SIZE”: “57cm”,

“URL”: " "

}

 

Here is the valid json I would expect:

 

[{

“ID”: 210000122824,

“NAME”: “Pivot Mach 6 Pro XT/XTR Float”,

“BRAND”: “Pivot Cycles”,

“EXPERIENCE”: “FS MOUNTAIN LONG TRAVEL”,

“SIZE”: “LG”,

“PRICE”: 7799.9900,

“PRICEWPROMO”: 7019.9500,

“DESCRIPTION”: “Pivot Mach 6 Pro XT/XTR Float”,

“COLOR”: “Stealth”,

“URL”: " "

},

{

“ID”: 210000110574,

“NAME”: “BBB Hotrock 24"”,

“BRAND”: “Specialized”,

“EXPERIENCE”: “KIDS BIKE, USED”,

“SIZE”: “”,

“PRICE”: 250.0000,

“PRICEWPROMO”: 250.0000,

“DESCRIPTION”: “BBB Hotrock 24"”,

“COLOR”: “”,

“URL”: " "

}

]

Hi @chris.odell1617208923357​,

Have you tried taking the generated SQL statement from the Matillion S3 Unload component and running it in the Snowflake interface itself? I find often times I can take the SQL statement run it in Snowflake and then tweak it until I get what I want. At that point I have a decision to make. See if the changes I made in Snowflake can be applied to the (S3 Unload) component or use the SQL Script component to execute the expected statement. I hope this helps in some way!

@chris.odell1617208923357​ Were you ever able to find a solution for this without reverting to CSV?

Hey there, thanks for responding, and yeah that's good advice especially since in this case the reason S3 Unload is producing this particular flavor of JSON is that Snowflake can only injest/export NDJSON. Having not previously worked with NDJSON I was confused as to why I wasn't seeing "standard" JSON.

 

To solve my problem I could probably have used Python to convert NDJSON to JSON before submitting to my vendor, but in the end we opted for CSV.

No, evidently Snowflake itself can only produce NDJSON and it didn't seem worth the hassle to try to convert that to regular json. Another "20/20 hindsight" moment with snowflake 😩