We recently addressed issues affecting SSO Logins and Box Notes. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.
Between between 07:42 AM and 08:47 AM PDT on September 14, 2022 , some users may have experienced difficulties while working in Box. During this time, customers may have experienced issues with SSO Logins and Box Notes. The issue occurred due to a failure in the network connecting one of our points of presence (POPs) to Box’s Application Data Center (ADC). We were able to resolve the issue by implementing backbone routing changes to bypass the affected network path. In addition, we are working towards migrating to the public cloud to leverage their diverse and global network to prevent similar issues from occurring in the future.
This issue was the result of a network failure affecting one of our POPs. These POPs possess two redundant transport circuits that are on diverse physical paths and are provided by two separate providers providing redundant connectivity. During this incident, the impacted POP experienced simultaneous failures of both of its redundant circuits due to separate service provider incidents.
Simultaneous failure of both transport circuits is a rare occurrence; in this case, the root cause was identified by the service providers as a “fiber cut“. This dual failure isolated the impacted POP from the rest of the network. In response, we implemented a temporary backbone routing change to resolve the issue at that time, which temporarily diverted the impacted traffic to a sub-optimal path. Later the connectivity was restored to the original preferred path when one of the circuit recovered. After that, the second circuit also recovered, thereby restoring the redundant connectivity to the POP.
The following corrective actions have been completed or are planned:
We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.
The Box Team