The API I need to extract from uses a pagination strategy that is a combination of a page number sent in the URL (for example: https://www.blahblahblah.com/api/endpoint?page=1) and a Response Header item that provides the total number of pages that exist for that query.
This seems to follow the strategy called "Page based" pagination within the API Extract / API Query profile setup wizards, however, within the API Extract or API Query profile setup, there does not seem to be a way to access the Response Header information from the API response in order to use it as the "Last page key" for the pagination config, so I am not able to handle the pagination strategy of this API. My goal would be to loop through all pages, starting from 1 until the value contained in a specific Response Header item. Another option might be to continue loading the next page in the sequence until a blank page of data shows up, then quit from the loop, but I can't tell how to make that work, either. My API extract runs through 1 page and exits without attempting to go to the next page.
Thanks for any assistance!