Hi everyone, I'm working on a POC to bring in IT/Infosec data from various vendors for analytics. One of the vendors that we're having trouble ingesting data from is Okta.
Currently, our IT team is gathering data from Okta via API and storing them in JSON files that are then stored in a an SMB fileshare. The problem is that the volume of data needed far exceeds the amount that can be gathered before being rate-limited. So each time my orchestration job runs, incomplete data is being pushed into our CDW --> meaning data in the target table is always lagging behind.
I'm hoping someone here has worked with Okta before and can provide some insights on creating a more scalable solution. Thanks in advance!