Your company is moving to Office 365, the tech division is going to be in charge of rolling out the change to your organization - read how...
Service Protection Throttling – Major Game Changer in O365
How Migration Timelines might Triple (At Least) Last December our team encountered a strange new behavior in our Office ...
How Migration Timelines might Triple (At Least)
Last December our team encountered a strange new behavior in our Office 365 onboarding projects.
Effectively we were seeing significantly reduced migration speeds to Office 365 due to “throttling-backoffs”. Despite having lifted the Microsoft throttling limits for various customer tenants, we encountered new limits that we have not seen before. This was a global phenomenon, spread across all projects and certainly not limited to a specific region.
This was not a pleasant situation as our logs were full of Office 365 Throttling Backoffs and migration performance was dramatically reduced. We initially assumed that this must be a bug/misconfiguration of the throttling policy and started opening support cases with Microsoft. Support concluded the same as the online throttling test in the Office 365 portal diagnosed: “the tenant is not being throttled” – the net result was that Microsoft support denied the fact that this was happening at all.
After working our various communication channels and providing some further backend diagnostics, we discovered we were not hitting any tenant-based throttling but something new known as “Service Protection Throttling.”
On Friday January 22, 2021, sources close to Microsoft Engineering confirmed that Microsoft put a new and permanent layer of service protection in place. Furthermore, the source states that those changes are permanent and that there is and will be no possibility of getting exceptions for tenants. Unfortunately, Microsoft did not deem it necessary to announce this publicly, but it was put in place to maintain database health and prevent mailboxes from becoming quarantined.
As the CEO of a migration company that offers cloud-based services, I can totally understand Microsoft’s points of view. A vendor that offers tenants in a shared environment needs to ensure that the performance of a single tenant does not affect the performance of other tenants. That said, there is no doubt this will have a significant impact on the Office 365 migration services market at all levels. Worst case, the new speed limits lead to a maximum ingestion speed of 150MB over a 5-minute slot across all mailboxes; Best case, to a similar speed per mailbox, migrating in parallel.
The net result is that the new policies could potentially triple or quadruple overall project times required to achieve the same migration as before Microsoft enforced the service protection throttling in Office 365.
Now for the good news!
While I may be biased, having the best team and technology in the industry allowed us to;
- React to the Service Protection Throttling changes & events very quickly and to find a solution that allowed us to get back to the desired speed levels without experiencing any throttling issues or bypassing valid throttling settings
- Solve this new challenge by dynamically scaling our microservices and Kubernetes nodes, up or down, depending on the actual load required
Here are some quick highlights to demonstrate how this played out in once we understood the challenge:
If you are currently performing a migration to Office 365 and you’re reading this, make sure to contact your migration vendor. Only they can solve the problem as Microsoft has a valid point in protecting other tenants from increased performance requirements caused by mass extractions and ingestions.
If you are about to start a migration project, always ask your vendor of choice for a POC before you commit, to make sure that they have the above described problem under control.
If you are a migration vendor and you’re reading this, you need to own this problem for your customers’ sake. When project times triple or quadruple, this becomes a severe issue for most of the customers and most likely threatens the project’s real ROI.
If you are Microsoft employee and you’re reading this, I understand and agree with service protection, but communicating these things internally (to your support) and externally (to your customers) would have saved an entire industry a lot of headache, time, and effort.
If you are a current (or future) Cloudficient customer and you’re reading this, be confident with the notion that we always have your back and do everything to solve issues as fast as possible.