JS Translation Regex leads to unexpected results and untranslatable strings

[thlassche]

Preconditions

Magento version: 2.1.2
Translate a string in javascript, for example like this (taken from mage/validation.js):

                    validator.passwordErrorMessage = $.mage.__(
                        "Minimum of different classes of characters in password is %1." +
                        " Classes of characters: Lower Case, Upper Case, Digits, Special Characters."
                    ).replace('%1', passwordMinCharacterSets);

The string should be for example concatted in JavaScript with the + sine.

Steps to reproduce

Look up the page where the translated string should appear (in this case: the register account page)

Expected result

The password error message is translated

Actual result

The password error message is not translated

Cause

The root cause of this, is the regex that selects JS strings for translation, as found in: /app/code/Magento/Translation/etc/di.xml. It's is very buggy and not fail safe. It skips every string that is not strictly formatted according to this regex. In this case the '+' sign causes the string to be skipped by the regex, but there are many more cases where this regex is skipped in JS files (e.g. iteration over parts of strings to translate them). See:

magento2/app/code/Magento/Translation/etc/di.xml

Lines 60 to 62 in 0be91a5

	<item name="i18n_translation" xsi:type="string"><![CDATA[~i18n\:\s*(["'])(.*?)(?<!\\)\1~]]></item>
	<item name="mage_translation_widget" xsi:type="string">~\$\.mage\.__\((['"])(.+?)\1\)~</item>
	<item name="mage_translation_static" xsi:type="string">~\$t\((["'])(.+?)\1\)~</item>

Looks like the implementation of the js-translation.json file should be a bit refactored.

Workaround

A workaround I use in production is to create a JS file which calls the$.mage.__() with the string that I would like to translate, e.g.:

    // HACK to make strings translatable, so the Mage Translate regex picks them up for JS translation
    $.mage.__("Minimum of different classes of characters in password is %1. Classes of characters: Lower Case, Upper Case, Digits, Special Characters.");
    $.mage.__("Minimum length of this field must be equal or greater than %1 symbols. Leading and trailing spaces will be ignored.");

[veloraven]

@thlassche thank you for your report.
Please, identify which version of Magento you are running.

[thlassche]

@veloraven 2.1.2

[danielasy]

This is the faulty regex: ~\$\.mage\.__\((?s)[^'"]*?(['"])(.+?)\1(?s).*?\)~

It is defined here: https://github.com/magento/magento2/blob/develop/app/code/Magento/Translation/etc/di.xml#L63

<type name="Magento\Translation\Model\Js\Config">
    <arguments>
        <argument name="patterns" xsi:type="array">
            <item name="i18n_translation" xsi:type="string"><![CDATA[~i18n\:\s*(["'])(.*?)(?<!\\)\1~]]></item>
            <item name="translate_wrapping" xsi:type="string"><![CDATA[~translate\=("')([^\'].*?)\'\"~]]></item>
            <item name="mage_translation_widget" xsi:type="string">~\$\.mage\.__\((?s)[^'"]*?(['"])(.+?)\1(?s).*?\)~</item>
            <item name="mage_translation_static" xsi:type="string">~\$t\((?s)[^'"]*?(["'])(.+?)\1(?s).*?\)~</item>
        </argument>
    </arguments>
</type>

It is also the root cause of #5509, #5820 and #9967, which are basically strings concatenated using + inside the __() function.

Another case where this check fails is if jQuery is used instead of $, making a method call such as jQuery.mage.__() fail.

[magento-team]

Internal ticket to track issue progress: MAGETWO-71380

[maaarghk]

I am seeing this problem too and trying out #10445 does not seem to fix it entirely. I am getting an error when compiling static files, core file i18n.js:

.Error while generating js translation dictionary: "" 
#0 /home/ukfstaging/www/htdocs/vendor/magento/module-translation/Model/Js/DataProvider.php(105): Magento\Translation\Model\Js\DataProvider->getPhrases('/**\n * Copyrigh...')
#1 /home/ukfstaging/www/htdocs/vendor/magento/module-translation/Model/Json/PreProcessor.php(86): Magento\Translation\Model\Js\DataProvider->getData('Magento/blank')
...

..see this regex101 paste for an example of a core file being messed up by this bug

As an aside this might also be the root cause of #5967 (MAGETWO-70599 apparently)

[magento-engcom-team]

@thlassche, thank you for your report.
The issue is already fixed in develop branch
But we will consider to backport the fix to patch releases

[rodrigowebjump]

Hi is there a plan to fix it in 2.2.x?

[hostep]

To add some more info to this ticket, because it is a long time ago since there was some activity here.

This problem was fixed in Magento 2.2.1 by commit: 27149cb
These changes were original proposed in #10445 which got included in 2.3-develop

I've now just created a backport of these fixes for Magento 2.1 over here: #14349

[magento-engcom-team]

Hi @thlassche. Thank you for your report.
The issue has been fixed in #14349 by @hostep in 2.1-develop branch
Related commit(s):

• 893e4f2f6b3e49a0b9c89fd312d79b98ae961daf

The fix will be available with the upcoming 2.1.14 release.