For more information about our Incident Response and Communications please read this support article.

We also maintain a list of Known Product Issues separate from this site here.

[Major] Issues with Watermarking

Incident Report for Box

Postmortem

We recently addressed issues affecting previewing of vector-based watermarked documents. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

Between Monday, July 10th, 2023 at 10:40 AM PDT and Tuesday, July 11th at 3:48 AM PDT, some users may have experienced difficulties while working in Box. During this time, users may have experienced intermittent preview errors for documents that had vector-based watermarking enabled. The issue occurred as the result of a recent update that caused a date/time formatting change that was not compatible with our watermarking service. We were able to resolve the issue by rolling back then change and patching the affected service to properly handle the date/time format change. In addition, we are working to improve detection as well as the robustness of handling these types of issues to prevent similar issues from occurring in the future.

Analysis

A system package which is responsible for handling internationalization and formatting of strings was updated. The new version of this package included significant whitespace related changes from the The Unicode Common Locale Data Repository (CLDR) which included change that introduced a Narrow Non-breaking Space (Unicode U+202F) in place of a standard space (Unicode U+0020). This character was not supported in the font that was used by the watermarking service which caused the watermark application to fail in the vector-based watermarking feature.

This incident revealed some areas for improvement in terms of robustness and testing strategy for this service.

Corrective Actions

The following corrective actions have been completed or are planned:

  • Ensure that we handle all expected unicode characters for date formatting as defined by the The Unicode Common Locale Data Repository (CLDR)
  • More robust handling of non-supported Unicode characters in general
  • More comprehensive testing to minimize the likelihood of similar issues occurring in the future.

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

Sincerely,

The Box Team

Posted Jul 12, 2023 - 11:34 PDT

Resolved

After further monitoring, this incident is now considered resolved. If you continue to experience any issues with Watermarks, please contact Box Support at https://support.box.com.
Posted Jul 11, 2023 - 03:52 PDT

Monitoring

A fix has been implemented and we're monitoring the results.
Posted Jul 11, 2023 - 03:07 PDT

Update

We continue to observe improvement, although intermittently. We are investigating what further work can be done to improve the current situation.
Posted Jul 11, 2023 - 02:23 PDT

Identified

We have identified the issue and a fix is currently being implemented.
Posted Jul 11, 2023 - 01:12 PDT

Update

We are continuing to investigate this issue and will provide more information as soon as it is available.
Posted Jul 10, 2023 - 23:57 PDT

Update

We are continuing to investigate this issue and will provide more information as soon as it is available.
Posted Jul 10, 2023 - 22:50 PDT

Update

We are continuing to investigate this issue.
Posted Jul 10, 2023 - 21:31 PDT

Investigating

We are currently investigating an ongoing issue where watermarks applied to content are not appearing during file preview or download. We will provide more information as soon as it is available.
Posted Jul 10, 2023 - 20:26 PDT