CaaS Link Truncation #2876
Replies: 10 comments 15 replies
-
What exactly is limiting us to 2k chars? Have we taken this limitation to MSFT via support case or feature request? Akamai supports URLs in the 8k range and chrome supports up to 2mb. Brotli compression will help a bit, but still probably not enough (18% savings with my test-case). For those not deep into milo/caas link lore... If the caas config tool can build its state using a real caas link instead of a json object, that would probably be the ideal fix. @chrischrischris, do you think that is feasible? |
Beta Was this translation helpful? Give feedback.
-
Adding a few more possible solutions here. A combination of these fixes will definitely fix/resolve the issue. I listed these in terms of easiest to hardest to implement (and left other possible solutions out), but relatively, most of these fixes are quite simple.
For example:
|
Beta Was this translation helpful? Give feedback.
-
Just an FYI since it's not inherently obvious -- compression algorithms like gzip already do this internally as part their algorithm and us doing this would give us no gain.
|
Beta Was this translation helpful? Give feedback.
-
@meganthecoder I believe I mentioned this in one of the earlier meetings related to this topic, but I am reiterating here for clarity. The character limitation issue can be consistently reproduced in the MS Word Desktop app by inserting a long URL and re-opening the link edit window. Upon doing so, the URL appears truncated. It does not seem that a session ID is required to replicate this behavior. Based on my previous discussions with Microsoft, they tend to prioritize addressing issues in the MS Word Web app rather than the Desktop version. In most cases, their focus is on ensuring that the Web app functions similarly to the Desktop app. If you haven't already contacted Microsoft, I can bring this matter up during one of the bi-weekly meetings I attend with them, to explore the possibility of having this issue resolved.
Do you have any insights on the expected gains from this? I assume this could be manually tested by modifying the CaaS JSON and applying encoding/gzipping techniques. Alternatively, maintaining a CaaS link map with Excel/JSON as a database might be a viable solution. Another approach could be a server-side solution (perhaps an AdobeIO endpoint) that uses a hashmap to return the CaaS link based on an ID. However, this would require modifications to multiple modules that handle CaaS to accommodate the change, not to mention the additional hop the page would need to make before getting the CaaS cards. |
Beta Was this translation helpful? Give feedback.
-
To Jed's point, I do think that there is a case to be made to Microsoft that imposing such a tight character limit seems arbitrary from the customer point of view. Given that the potential exists for much larger URLs, it hardly seems unreasonable to suggest they increase it. But as Sunil points out, we need to be prepared for the fact that we're most likely asking them to change it in two products, even though we only care about one of them. |
Beta Was this translation helpful? Give feedback.
-
Other than microsoft fixing the issue, we'll always have problems when many filters and/or tags are used. Things that might help: Low Effort:
we could do:
Medium Effort:
The unicode string would then have to be added to the url as a hash manually and NOT using High Effort:
|
Beta Was this translation helpful? Give feedback.
-
Hi @meganthecoder, Here is the full implementation of converting switching from base64 to base 85. Adding this should cut your URL lengths on average by ~450 characters (check above gist). This plus adding brotli compression as suggested above should help cut it down even further. Below is the high level idea for why this algorithm works case you run into any issues or would like to make any further enhancements.
Encode this number in Base64 and Base85 to get the specified output. Base64 Encoding:
Base85 Encoding:
Comparing the results:
This gain is always guaranteed to be the case as Base85 is more efficient than Base64. Base85 represents 4 bytes of binary data with 5 ASCII characters, whereas Base64 uses 4 characters for 3 bytes. This results in about a 20% size reduction compared to Base64. Some additional notes: The sample config I used is much more complex and has many more edge cases than anything that an author could possibly set up inside BACOM (so the implementation should be very robust)
For the page you are having issues w/ (the main resource page w/ 2940 characters), an authoring fix would be to use automatic filters versus custom filters (as using custom filters adds a lot of bloat to the config). That should also cut about ⅓ of the characters in the URL. Good luck! |
Beta Was this translation helpful? Give feedback.
-
Could use even more characters - Base256 Encoding: |
Beta Was this translation helpful? Give feedback.
-
Hi @meganthecoder, Here is another potential solution that solves this problem and is significantly simpler than the base85 approach and gives more gain: This alone should solve your problem and you should be able to keep the majority of the original Milo implementation in place and not have to worry about encoding issues. If you want more compression than this, you can add base85 but I think it's overkill. Solution: Use Dynamic Key MappingObjective: Reduce the size of the JSON configuration string by replacing long property names (keys) with shorter ones before compression and encoding. We traverse the configuration object to collect all unique keys used at any level in the object and swap to use this mapping.
Results
|
Beta Was this translation helpful? Give feedback.
-
Thanks for everyone's help and input on this! We've solved the immediate need for bacom for now and gotten the longest link down to 1968 characters. Yay! What was implemented:
That being said, it's still relatively close to the limit. If there are any changes to the config, we could find ourselves over the limit again and might need to reopen this discussion. With that idea in mind, another option that we might consider for the future is to focus on reducing the filters specifically since bacom's custom filters add about 1450 characters themselves. We can't use automated filters to get what the PM wants as is, but maybe enhancements could be made in the future. |
Beta Was this translation helpful? Give feedback.
-
Problem:
During bulk updates on bacom, some of the caas links got truncated because they were too long (over 2000 characters). This broke the caas cards on those pages and required manual authoring to fix.
Background:
From what I can tell, we had an issue with truncation during the loc process last year, and @chrischrischris implemented gzip compression for caas links in Oct 2023. PR #1401. Ticket: MWPW-135873
However, even with this compression, we have links over 2000 characters. For example, the caas link on the bacom resource page is 2940 characters. A lot of bacom's new resource topic pages are over the limit because they have a lot of customization.
Ideas on how to solve:
gzip
todeflate-raw
will only reduce the length by about 2%, which isn't enough.state
in the config url or could we include only what options changed from the default?The minimum caas url length is 1247 characters (with the default options). If the Microsoft Word max is 2077, that means that we're using more than half of our space with boilerplate. Generally, I'm on the side of making names and labels human-readable, but in this case, it might be worth it to abbreviate the param names to save space.
To note: We're also looking into this on the Microsoft side, but considering how slow that process can be, we're focusing on changes we can make in milo.
Beta Was this translation helpful? Give feedback.
All reactions