I'm using Matillion ETL for BigQuery and the Database Query component.
My SQL query is select * from [mydatabase].[dbo].[mytable].
As I watch the the task run, it is staging 1000 records and then giving error "Sorry. You can't add more items to a closed queue."
If I limit my select to top 1000 or less, it runs fine. It's only when I go above 1000 records that I receive the error.
I have other orchestrations using the Database Query component, connecting to the same database, doing a similar SQL query but a different table. These are running without issues.
Could someone help me troubleshoot this error? I'm a new to the forum, so please let me know what additional information I can provide that would be helpful.
My original table in SQL had more columns than my destination table in BigQuery. I was doing this on purpose, but the select * must have been confusing something. If I select the specific columns that are in my destination table I'm good to go. Hope someone else finds this useful.
Hmm… that’s interesting. It does sound like you've figured it out. But, I’ve not seen a case where select * causes a problem. I wonder if when you switched to explicitly naming the columns, if you omitted a column that was the cause of the error. Other than that, I'd call it resolved.
If you are still having an issue, I'd suggest you raise a support case so that we can look at your server log.