Microsoft Graph - Don't Get Throttled!

The Microsoft Graph endpoint is normally highly performant. The infrastructure behind Microsoft 365 will allocate computing resources based on demand to ensure that periods of high traffic levels do not result in degraded performance. In addition to dynamic scaling of resources, another mechanism that Microsoft uses is throttling. If you make too many requests then you will start to get HTTP 429 (too many requests) response codes returned. Not all resources are metered and while this initially applied just to the Outlook APIs, we can assume that more resources will be metered over time.

How many requests is "too many"?

So the obvious question is: what is the definition of "too many requests?". The answer to this is that you will get throttled if you make a number of requests that would be detrimental to the service as a whole, which isn't actually that helpful. A figure that has been stated publicly is 10,000 requests in a 10-minute period. This applies on a per-user and per application ID basis. So you could make 10,000 requests on behalf of each user or group in your tenant in a 10-minute period before triggering the throttle, and if you exceed the threshold for a particular user then it is only that user that gets throttled. If you are using app-only permissions then that counts as a single user.

This is all well and good, but for practical purposes you have to accept that these rules could change at any time, and in addition you could well find that a lower threshold kicks in if the service is becoming overloaded for whatever reason. You might also find you get throttled if you make a large number of requests across your tenant, the 'rules' above notwithstanding. The reality is that unless you are making a small number of requests, you should design your application on the assumption that you could get a 429 response at any time and build in logic to deal gracefully with that situation when it arises.

If your application is throttled you will get a Retry-After response header in addition to the 429 response code, telling you the number of seconds your application needs to wait before making more requests. You also get a descriptive Rate-Limit-Reason header (I hesitate to call it an error message) which you probably will want to log rather than show to a user. Obviously you will want your app to wait for the Retry-After period before making another request, otherwise you will just get more 429s and probably extend the throttling period even further.

Avoiding Throttling

Clearly it is preferable not to be throttled in order to keep your application performant. There are a number of strategies you can use:

• Don't do automatic retries as these will accumulate requests against any throttling limit.
• Retrieve multiple items in one request, e.g. get a page of list items rather than retrieving them individually, and use $select to avoid getting properties you don't need to keep the payload down.
• Increase the value of $top to get larger pages (default is 10 but maximum is 1000)
• Cache data locally where needed to avoid re-fetching data, while being careful not to breach data privacy, retention policies and terms of use of the Microsoft Graph
• Use webhook notifications rather than polling requests to detect changes (you can also use delta queries to reduce the response payload).
• Use JSON batching to combine multiple requests into a single batch request.

You should think of the Retry-After period as a recommendation. If you re-try the request after the Retry-After time has elapsed it is still possible that you will get a 429 response. This means that throttling is still in effect. In this case you should again wait for the (possibly updated) Retry-After period and try again, repeating this process until the request succeeds.

It is worth noting that not all of the Microsoft Graph APIs provide a Retry-After header. It is possible for an API to have throttling implemented but not yet support the Retry-After header. In this case a strategy would be to have an initial wait period and keep increasing it exponentially with each 429 response.

More guidance is provided in the Microsoft Graph documentation.

Microsoft Graph Data Connect

The Microsoft Graph API is designed for transactional access to Microsoft 365 data on a user-by-user basis. Some applications really require bulk access to data, for example aggregation of data across a tenant, fraud detection, analysing resource utilization and efficiency or building organizational directories, to name a few use cases. Typically these types of processing occur on a scheduled basis, for example every night.

In this case the Microsoft Graph API endpoint is a very inefficient way of doing this and the application is likely to encounter throttling limits. A better solution in this case is to use a new feature (still in preview at the time of writing) called Microsoft Graph Data Connect. It allows you to get at the same data that's available through Microsoft Graph APIs (currently only a limited subset is available), but in a scalable way. It also allows you to build sophisticated analysis pipelines using tools like Azure Data Factory. All this processing occurs within the Azure data centre for better data protection.

For more information see the Microsoft Graph Data Connect documentation.