Magento 2.3.4 JS-Translations fall back to default store language

[jhruehl]

Edit by @DrewML: Bug has been fixed, but a more thorough solution should still be implemented. See later comments in thread for more details

Original Issue

In a multistore setup with different languages, some translations like for "Total" in checkout cart or the ones for the "add to cart" button, just after clicking it, fall back to the ones given for the default store view.

This only happens with the Baler module being enabled.

A colleague of mine debugged it to magento/module-translation/view/base/web/js/image-translation-dictionary.js

This was introduced in the following commit:
magento/magento2@71c781d
Which was seen as a resolution for
#13

[DrewML]

cc @adifucan - any ideas?

[jhruehl]

Update: After I reverted most changes of the given commit, it works again.

[adifucan]

I confirm that JS translations with multistore setup and different languages do not work with Baler enabled. Looking into this issue.

[adifucan]

@DrewML Looks like the main reason for this issue is that non-US locale dictionaries are merged into US core-bundle.js.

For example, I have 2 stores with 2 locales: US & DE. After setting up baler I see German dictionary merged into pub/static/frontend/Magento/luma/en_US/balerbundles/core-bundle.js. As a result while surfing US store I get German JS translations.
As far as I can understand DE dictionary should be available only in de_DE/balerbundles/core-bundle.js not in en_US/balerbundles/core-bundle.js.
So baler does not know about it while generating core-bundle.js files.

[DrewML]

Thanks for the report @jhruehl, and thanks for digging in @adifucan.

Based on what @adifucan showed me, I believe I know the root cause and how we regressed this.

Before

In Magento core, we used to fetch js-translation.json at runtime via a script in the body that did a request at runtime. baler could not see this script, so it never even attempted to bundle js-translation.json.

This had the unfortunate problem of race conditions, though, since js-translation.json wasn't expressed properly as a dependency in the graph. So we fixed the race!

After

Once the fix was made in core, js-translation.json starts being pulled in via a dependency to a define call in mage-translation-dictionary.js. Once this changed happened, core-bundle.js started to include translations (which is a good thing).

Problem

The problem here is a short cut that I took in the code to save time (execution time and development time).

Similar to how the quick content deploy strategy works in Magento core, baler does a similar trick: It does most of its work on a single theme, and then copies that to other locales.

Unfortunately, we're not doing anything special to handle js-translation.json, so the copy from the first locale gets passed to all locales for that theme.

Solutions

I can think of 2 possible solutions. One is fast, one is thorough.

Fast
We can just exclude js-translation.json from being bundled. RequireJS at runtime should fall back to a network call, and nothing breaks.

Thorough
We can special case the handling of js-translation.json. Instead of having 1 core bundle that we copy to n locales, we'd take the shared graph, and bundle the graph once-per-locale, changing the js-translation.json node in the graph each time to be linked to the proper localized file on disk.

[DrewML]

For the Fast solution mentioned above, we can take care of the ignore in computeDepsForBundle.ts, which is the same place we exclude require built-ins. Would allow us to keep a complete graph (useful for debugging and other tooling), but make sure the translations aren't in the bundle.

[boldhedgehog]

@DrewML please go on with the Fast solution or make the exclude list somehow configurable.
We have a custom Magento module that allows to add translations via the backoffice, and the js-translation.json file is generated dynamically, then a rewrite in nginx configuration server this generated file on request.

PS: Today I debugged "the endless loop" issue with our theme, and I think it fails because js-translation.json file is not found:

"ENOENT: no such file or directory, open '/var/www/html/pub/static/frontend/ISM/improved-luma/en_US/js-translation.json'"

In internal/util.js:288

Once the file is created, baler continues and finishes the build.

So, I assume, reading file exception/error handling is not fine.

[DrewML]

@tdgroot submitted (and I merged) a PR that excluded js-translation.json for now (aka the "fast" fix), and we'll leave this open until we do the "thorough" solution.