Advice regarding bulk processing with API. Is there a limit?

API
1

Hi!
Been using the API to pull company data but having issues as we seem to be hitting timeouts or being kicked off before finishing the process. This is the process we are doing:

  1. Getting all companies via:
    /v4/f/{root_firm_id}/companies

  2. Getting latest period for each company via:
    /v4/f/{root_firm_id}/companies/{company_id}/periods

  3. When we have the latest period we are fetching all reconciliations in that period for each company via:
    /v4/f/{root_firm_id}/companies/{company_id}/periods/{period_id}/reconciliations

  4. After this we look through all the reconciliations for the result that returns the handle that matches the reconciliation we are looking for and grab its id

  5. Then we fetch that specific reconciliation for that specific company in that specific period via:
    /v4/f/{root_firm_id}/companies/{company_id}/periods/{period_id}/reconciliations/{reconciliation_id}

If we do this process with all of the ids hardcoded it works fine, however doing it in bulk it seems to not finish at random points and then it destroys the access_ and refresh_token. Are we hitting some sort of API limit? If so what is the limit? In what sort of timeframe?

Many thanks

2

Hi,

Do you happen to know how much time it takes to run that entire process in bulk?
An access token only lasts for about 2 hours, is it possible that the entire process takes longer than that?

Currently there is no API limit, so my guess would be that your iteration through all your companies and periods might cause the access token to get expired which will stop the request process.

Any chance you can have a look at the time it takes until you encounter that issue?

Kind Regards

3

Hi,

Thanks for the response, we haven’t been able to run the entire bulk yet so I can’t give you an answer to that… Our longest runtime so far has been 50 odd minutes before it kicks us off but the kicking off is seemingly random. One time stopping at 25 minutes, another at 15 minutes, etc.

We get a new access and refresh token at the start of every request so it shouldn’t be that either since that access token should still be active… Couldn’t find any mentions of any API limits written anywhere so thought would be best to at least ask to eliminate that as a possibility.

Thank you and Kind regards

4

Hi,

We’ll have to investigate this more thoroughly with our own monitoring logs.
Currently there is no limit set in our API nor is there a mechanic that would invalidate your tokens if you were to perform too many requests.

Can we potentially pick this up through e-mail?
If so, could you send us some timestamps on which you had such occurrence(s) happen in your API process to api@getsilverfin.com?
That way we can investigate this issue that you’re having with our own monitoring logs, to hopefully find the root cause for this.

Kind Regards

5

Hi,

I’ve sent an email including logs with subject line referencing this forum post.
We managed to have one successful bulk but still puzzled as to what is causing the other failures, and why it is failing so consistently.

Many thanks and Kind Regards