I'm having an issue when I pass parameters to an API that requires link paging. I can sample the data and retrieve data up to the page limit. When I run and it goes over the page limit I get a bad request error. If I remove parameters then I have no problem getting data from all the pages.
I am pretty sure what is happening is Matillion is adding the parameters onto the link, however this API does not accept those. I have no idea how to ensure the RSD includes the parameters on the initial request, but not on requests from the link. Any guidance?
Hi JS217889,
I think Matillion have implemented Link Header paging in such a way that it demands a page parameter too. In the API Query and API Extract dialogs it's called a Page Size Parameter and a Page Size. You can't get through the dialog without supplying a value.
So every time the paging is invoked, the address that gets called always has an extra URL query parameter on the end. That's maybe the problem you are hitting.
If you're using an API Query this document might help you hand-code your own. There is a section on Link Header Pagination, and it doesn't force you to add a URL parameter.
BR
Thanks for the input! I was able to get around this through using a variable that wasnt a URL parameter. This solved my issue so the parameter gets added in the initial call, but not in the subsequent link calls.