Meta description and keywords transform to html entities for non latin/cyrilic characters in category and product pages

[fouli86]

When setting up meta keywords and description in the categories and in every product, all non latin characters transform to html entities, but not the title.

Preconditions

1. Magento CE 2.1.8
2. PHP 7.0.22
3. Server version: MySQL 5.6.35
4. el_GR language pack or any other non latin language

Steps to reproduce

Choose and enable a non latin language pack, ie Greek

1. setup the meta keywords and description for a category and a product
2. view page source

Expected result

All keywords appear as they should and category pages are "read" by google

Actual result

All non latin words are html entities

I have double checked the data table entries, everything looks as it should.

Note: The meta title is shown correctly, but also everything works fine on the homepage.

[korostii]

For reference: this bug has been reported numerous times before.
Please see the following issues, for example: #5299, #4861, #2295
Still not fixed, it seems.

[magento-engcom-team]

@fouli86, thank you for your report.
We were not able to reproduce this issue by following the steps you provided.
Please provide more details regarding your environment, or try to reproduce this
issue on a clean installation.

[fouli86]

Hello,
Database Server:
Server: Localhost via UNIX socket
Server type: MySQL
Server version: 5.6.35 - MySQL Community Server (GPL)
Protocol version: 10
Server charset: UTF-8 Unicode (utf8)

Web Server:
cpsrvd 11.64.0.39
Database client version: libmysql - 5.1.73
PHP extension: mysqli, curl, mbstring
PHP version: 5.6.30

I am using the language pack found here
I can't think of other details to provide you. Please do ask me for anything more specific. I haven't got the time yet to upgrade to the 2.1.9 version, in order to check if the problem still exists. I can send you my url in a private message or something, so that you can all better understand the issue.

Thanks in advance

[magento-engcom-team]

@fouli86, may you, please, describe "expected result" more clear

[fouli86]

Sure! Let me be more clear with the following print screens:
Here is the Search Engine Optimization of a category on the backend

The expected result would be:

But....the actual page source is:

All words written in Greek are HTML entities, but note that the title is shown correctly (in greek characters). I have also noticed that whenever a category has no keywords and description, it uses correctly the keywords of my homepage.

I'm afraid I can't be more clear, but I really appreciate your help.

Thank you

[orlangur]

Assuming https://webmasters.stackexchange.com/questions/76522/are-html-encoded-symbols-allowed-in-meta-description and https://webmasters.stackexchange.com/questions/76211/meta-description-has-html-entities-will-google-show-the-real-character-or-the-e why is it a problem?

[hostep]

@orlangur: because it causes the html to be larger in filesize then if the characters are outputted just as is, in utf-8 format.

In case anyone is interested, the entity encoding happens on this line:
https://github.com/magento/magento2/blob/4b56264/lib/internal/Magento/Framework/View/Page/Config.php#L229

I think the call to htmlentities should get removed here, it's not the task of this class to already start encoding text. If there needs to be some encoding/escaping it should happen when outputting the html, so in a Block or in a phtml file.

[magento-team]

Internal ticket to track issue progress: MAGETWO-83024

[magento-team]

Hi @fouli86. Thank you for your report.
The issue has been fixed in #12955 by @dverkade in 2.3-develop branch
Related commit(s):

• c49c52a

The fix will be available with the upcoming patch release.

[magento-team]

Hi @fouli86. Thank you for your report.
The issue has been fixed in #12956 by @dverkade in 2.1-develop branch
Related commit(s):

• 85db3a0
• 66d4d4e

The fix will be available with the upcoming patch release.