I hope you don't mind, I had a read over your post and thought it would be something to raise with our internal team, who are actually going to take a look at this and test some methods out internally and share what they find out!
It would be brilliant to learn if other users have experience of using this and what methods they find to be a success!
I believe we do support entering multiple values for "attributes" using a comma delimited list.
However, if I understand your issue correctly, the answer to your question is: "no, we currently don't support multiple parameters with the same name."
Is this requirement (i.e. separate parameters of the same name) a common one or is it specific to that API?
Would you be able to share the documentation of how this API is to be used, and thus, why separate parameters are required?
Also, understanding how those parameters should be handled will help us come up with a better, more relevant, solution.
I agree that the API Extract profiles in their current state, cannot be built to handle the list of attributes using the method you pointed out. There may be another way of accomplishing it though. Have you tried the same approach with an API Query Profile? I have spent a bit of time in that part of Matillion and I don't see why you couldn't accomplish the same thing using an API Query Profile. You can set the URI to whatever you would like. If you wanted to dynamically build the attributes list portion of the URI outside the actual API Query component and then pass it in as a single parameter, you could use that parameter to tack it on to the end of the URI. This should be pretty straightforward.
As an example your API Query Profile would look similar to this:
With the "recipients" request I get the list of all customers and I have to pass a list of attributes I want to get as response.
Yes I know there is an "AllAttributes" parameter but in this case this parameter is not working.
So the current approach is to pass all the parameters. This is working with the api query profiles without any issues.
In this case I think the actual issue is that the extract profile tries to translate the provided url and will for example extract all the parameters to the list at the bottom. That is nice but might be an issue in some cases. Maybe a quick win that will also fix other issue: I it possible to add some kind of override?
So in this case I only want the exact url passed and don't want to interact with any parameters.
I just need to handle the paging afterwards but I don't need the parameters inside a special list to interact with them.
Actually I have already a "old" solution in place that was build over 1 year ago using only the rsd files.
We also have even some more sophisticated requirements. We try to build our solution as generic as possible to avoid the same logic twice. So in this solution we are calling the same API but for different brands using external variables that are passed through the api using "connection options other" . And even inside these brands is another generic approach to call different parameters that are handled using an external table.
The main reason I started migrating these jobs is that they are pretty "old". I like the look and feel of the new approach to build such api connectors. It is a lot easier and makes more fun to use. RSD files tend to be pretty annoying..
But like always the details are an issue, so I will stick with the old solution for now.
But maybe you can clarify something from an customer point of view.
Do you actually understand why I should use Extract Profiles instead of the Query Profiles?
What I understood so far. The Query Profiles are more or less legacy and are using rsd files.
The Extract Profiles seems to use another technology at the backend and are not using rsd files anymore. But with the extract profiles I have the limitation of some missing features/bugs and I cannot edit the actual "code".
I think the extract profiles are the new approach and the query profiles will be deprecated soon?
But in my opinion the preferred approach would be to mark them as deprecated as soon as possible and invest more time into the development of the extract profiles. Having both in place without any added value will just make stuff for everyone (customers and Matillion) more confusing and complicated.
Thanks for the extra info. Yes, I agree the Extract Profiles seem to be the new approach. I haven't heard officially if Matillion is planning to migrate away from the RSD form but it definitely seems a likely path. The one part that I don't know if they will be able to solve and they may not need to is being able to do the "If then" type logic within the Extract Profiles like you can do in the RSDs currently.
I do agree though there is definitely room for improvement on the Extract Profiles. I think you should drop an Idea on the Ideas Portal in regards to the list type approach form Parameters. That seems to be a standard to a certain extent in many APIs. If Matillion is in fact shifting from RSD's to Extract Profiles I agree, the sooner they mark the feature deprecated the better. Thanks for sharing your thoughts and issue!
