We also maintain a list of Known Product Issues separate from this site here.
We recently addressed issues affecting uploads & downloads. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.
Between 4:50pm and 5:12pm on February 1st, 2021, some users may have experienced difficulties while working in Box. During this time, users would have experienced increased latency while attempting to upload or download content. The issue occurred as a result of a change that was made by a 3rd party partner. We were able to resolve the issue and the impact to Box by rolling back the change. In addition, we are working with our 3rd party partner to improve processes around communications for changes with high customer impact to prevent similar issues from occurring in the future.
Analysis
One of Box's 3rd party partners rolled out a change to one of their rollout tooling which impacted request routing to the partner. The resulting traffic overloaded the services resulting in the latency and errors. While some of the impact was mitigated through an automatic fallback to route the requests to other partners, ~~but~~ it wasn't completely mitigated until the 3rd party partner rolled back their changes.
Corrective Actions
The following corrective actions have been completed or are planned:
Added additional monitoring and alerting within partners infrastructure to detect these issues in the future
Our partners addressed the issue by improving their rollout process for all all changes to automatically detect and revert this type of issues thus avoiding such outages.
We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.
Sincerely,
The Box Team