[Major] Customers are experiencing impact to Public API, Uploads, Downloads, and Metadata.

We recently addressed issues affecting the customers for Downloads, Uploads and Public API services. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

‌

On January 31, 2023 between 12:02 AM PST and 12:27 AM PST and on February 9, 2023 between 11:03 PM PST and 11:28 PM PST, some users may have experienced difficulties while working in Box. During this time, some users may have experienced issues with Logins, Notes, Downloads, Uploads & Public API services.

‌

Analysis

On the day of the incident (Jan 31st 12:02 AM), it was observed that an internal Garbage Collection process triggered; which caused the system to pause from responding to services hosted on it. This caused a high internal queue build up as the system backlog built due to CPU resource contention. As the queue size build up was significant, the recovery took some time.

‌

Corrective Actions

The following corrective actions have been completed or are planned:

• We are working towards reviewing and adding more alerting for improved diagnostics for such issues.
• We have added a caching layer to reduce the dependency between our backend datasets and customer applications. This should reduce the potential impact to the customer experience if we face a similar issue.

‌

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

Sincerely,

The Box Team