For more information about our Incident Response and Communications please read this support article.

We also maintain a list of Known Product Issues separate from this site here.

[Major] Issue with Uploads, Downloads, Logins, and Public API
Incident Report for Box
Postmortem

We recently addressed issues affecting uploads, downloads, logins, and public API. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

On May 17, 2022 between 11:05 AM PDT and 11:07 AM PDT, some users may have experienced difficulties while working in Box. During this time, uploads, downloads, logins, and public API experienced some failures or errors.

Analysis

The issue occurred when the leader node of an internal clustered system failed a soft restart. During a maintenance window, an internal infrastructure system that uses a leader-standby clustering model failed to elect a new leader. As a result, some users experienced issues with uploads, downloads, logins, and public API. Additionally, during the impacted timeframe, internal client services of the system did not rely on cached data as a fall-back, which exacerbated the issue. We were able to resolve this issue by issuing a hard restart of the affected node. We are continuing to investigate to determine the root cause of the issue and are developing additional corrective actions to prevent similar issues from occurring in the future.

Corrective Actions

The following corrective actions have been completed or are planned:

  • Performing a hard restart of the affected node to return it to a normal state.
  • Identifying and addressing the underlying root cause of the cluster leader-election issue.
  • Improving the clustering configuration to permit making a cluster passive before maintenance.
  • Conducting further analysis to determine why internal services did not utilize cached internal data during the impacted timeframe.

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.

Sincerely,
The Box Team

Posted May 31, 2022 - 14:01 PDT

Resolved
From approximately 11:07 AM to 11:14 AM US Pacific time, customers may have experienced issues with Box’s uploads, downloads, logins, and public API. Our systems automatically detected and corrected the underlying issue. If you are experiencing any issues with these services, please contact Box Support at https://support.box.com.
Posted May 17, 2022 - 11:00 PDT