We recently addressed issues affecting the User Events API. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

Between 10:00am PDT and 10:40am PDT on April 23, 2022, some users may have experienced difficulties while working in Box. During this time, the User Events API may have experienced increased latency or failed to respond. 

Analysis 

The issue occurred as a result of an internal database cluster failure.  The Box User Events API retrieves information from a distributed database cluster that stores User Event data.  This distributed database cluster utilizes a common orchestration system to control various aspects of the service.  During the impacted timeframe, the orchestration system became degraded due to resource contention issues; as a result, the distributed database service was negatively impacted, and the User Events API experienced delays returning responses to user requests.  We were able to resolve the issue by migrating traffic to our standby database cluster.  Upon performing failover operations and restarting the impacted systems, we were able to successfully restore the service. In addition, we are reviewing the speed of our failover procedure and cause of the database cluster failure to prevent similar issues from occurring in the future.

Corrective Actions

The following corrective actions have been completed or are planned:

• Deploying configuration changes to the coordination service to minimize the risk of recurrence of this issue. 

• Updating failover code and procedures to reduce overall failover time.

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

Sincerely,

The Box Team