We recently addressed issues affecting Box Drive users. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

Between 2:55am and 9:20am PDT on March 29th, 2023, some users may have experienced difficulties while working in Box. During this time, some users may have experienced delays in Box Drive synchronizing changes. The issue occurred due to a regularly scheduled automatic internal credential rotation failing to propagate completely. We were able to resolve the issue by manually synchronizing the affected service with the latest credentials. In addition, we are ensuring services self-validate against the credential freshness constraints to prevent similar issues from occurring in the future. 

‌

Analysis

Key rotation is one aspect of the broader key management and a component Box’s overall security system, which requires key holders to update their keys regularly before they are revoked. In the event an automatic key rotation occurs and a key holder does not refresh their credentials within their valid lifetime they will no longer be able to authenticate and receive authorization to communicate with external systems. In the case of this issue, a single internal service responsible for managing realtime events did not update its credentials following an automatic rotation and thus not able to make those new events available to our customers.

‌

Corrective Actions

The following corrective actions have been completed or are planned:

• Add an alert to page on-call engineers should the affected service fail to update its credentials to enable a user to investigate and manually complete the task
• Update the affected service to add an additional automatic check for credential freshness to watch for rotations and update automatically

‌

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

Sincerely,
The Box Team