The first time the query runs I get a page worth of data back in about 5 seconds. I am using a loop iterator to make the calls for subsequent pages. Each time the loop iterates it takes longer for the same number of records to be returned. By the time I get to the 80th iteration it takes almost a minute to call/return the page of data! Soon after I start getting time out errors and the pipeline fails.
This is a new issue. When the pipeline ran last week it did so without failure and each iteration took between 5-8 seconds.
Unfortunately, the failed API Call tasks are showing the message: “The log file has been truncated because it’s too large. If you require a full size logfile please visit our support website for full instructions.”
For starters, can someone please explain how I can get the full size log file? I’ve looked through the docs and am getting quite frustrated trying to find something relevant!
I don’t know whether the API issue is with our IT infrastructure or the API Vendor, so it would be good to have proper logs to try to determine that.
I guess it’s also worth asking, has anyone seen anything like this with the API Query component?
Thanks for your post, first of all, the feedback with regards to docs, I have taken this away and passed it onto the team who have already raised this internally, they are brilliant and we should see some changes there to reflect this. So thank you for raising that.
When it comes to reviewing logs, its really difficult to do so from the community, the best option would be to raise a support case and our team can take a look with you and resolve the issue.
In the meantime if you need anything else, please let me know.
My IT colleague wasn’t able to uncover anything. He also had a look at the .rsd files for the API query, but couldn’t see any issue there either (not surprising as the query was working perfectly well last week).
However, he did say that at one point he overwrote something in the .rsd, so to make sure he didn’t cause additional problems he pasted the original back in (which he’d saved, fortunately).
Lo & behold the pipeline ran successfully overnight!
I have no idea whether re-saving the .rsd had anything to do with it (can’t see how), or what else may have changed. Again, no sign of any changes on our side of things.
I hate intermittent issues that don’t clearly identify the problem … but I’m glad it is resolved - for now at least (fingers crossed).
Thank you for sharing the update. I’m glad to hear that you’ve managed to get the pipeline running again. We appreciate any feedback, and I have already passed this on to the team for you.
If you ever need anything, please do not hesitate to post.