Previous incidents

3/26/2025 10:30AM MST - Our API endpoints and some internal systems were heavily stressed by unplanned load from services. We have scaled systems to relieve pressure, and are adjusting some rate limiting to reduce risk exposure. Systems are normalizing but will take some time to process out.

3/26/2025 3:30PM MST - Systems are recovered and we are closing this incident.

We experience an unusually high volume of API calls in USA that over-running current scaling, causing some API calls to fail. Investigation and remediation on the way.

12:12PM MST - Systems have stabilized. We are still heavily engaged in monitoring and root causing the issue. More details as we have them. We are not considering this incident over just yet.

6:28PM - Systems have been stable for some time now. Root cause was issues that arose from migration of a backend system. Issues resol...

Situation : We are experiencing an issue with the reliable-tasking extension.
Task : We are effecting hardware upgrade that we expect to stabilize and resolve the issue.
Expected Impact: There will be a loss of currently buffered reliable tasks.

Jan 24, 2025 10AM - Issue has been stabilized, but we are still root causing the issue. No expected Impact at this time. We will update as we have more information.

Jan 24, 2025 5PM - We have implemented a fix, and while we expect resolution, will ...