Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EZP-28950: Add support for utf8mb4 charset to i18n #1361

Merged
merged 1 commit into from
May 28, 2018
Merged

Conversation

emodric
Copy link
Collaborator

@emodric emodric commented May 23, 2018

https://jira.ez.no/browse/EZP-28950

eZ legacy does not recognize utf8mb4 charset, instead falling back to conversion to a single byte charset set by eZTextCodec, which results in the following beautiful error messages:

Query error (1064): You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use near '???????????????????????????????????????????????????????????
?????????????????????' at line 1. Query: ?????????????????????????????????????????????????????????????????
??????????????????????????????????????????????????????????????????????
[ May 23 2018 10:47:03 ] Unexpected error, the message was : Call to a member function
attribute() on null in vendor/ezsystems/legacy-bridge/bundle/LegacyMapper/Security.php on line 97

The effect of this PR is that utf8mb4 is internally recognized as utf-8, resulting in no conversion from UTF8 taking place when used for example by the database layer.

Disclaimer: I'm not claiming that this is a proper fix, especially in this obscure part of legacy, but it seems logical :)

@glye @andrerom @gggeek @alongosz Any input on your side?

@glye
Copy link
Member

glye commented May 23, 2018

Note: This is a prerequisite for utf8mb4 to work on legacy, but doesn't stop the old utf-8 from working. Should be backported to all supported branches.

@gggeek
Copy link
Contributor

gggeek commented May 23, 2018

+1

@glye
Copy link
Member

glye commented May 23, 2018

For reference, before this PR, legacy doesn't know what utf8mb4 is, and assumes it is a singlebyte encoding, which obviously fails. See

After the fix, this returns utf-8, which means that input and output is the same, and eZTextCodec does nothing to our SQL strings, which is what we want. utf-8 in PHP is fully compatible with utf8mb4 in MySQL. See

if ( $this->InputTextCodec )

@micszo micszo self-assigned this May 25, 2018
Copy link
Member

@micszo micszo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified with sanities.

@micszo micszo removed their assignment May 28, 2018
@glye glye merged commit c1d42f7 into master May 28, 2018
@glye glye deleted the utf8mb4_support branch May 28, 2018 08:03
@glye
Copy link
Member

glye commented May 28, 2018

Thanks @emodric & @micszo! Merged.

@glye glye removed request for andrerom and alongosz May 28, 2018 08:04
emodric added a commit to emodric/ezpublish-kernel that referenced this pull request Jun 28, 2018
Due to utf8mb4 changes, only ezsystems/ezpublish-legacy version compatible is 2018.06, to be released soon. All other versions will fail with the error described in ezsystems/ezpublish-legacy#1361
andrerom pushed a commit to ezsystems/ezpublish-kernel that referenced this pull request Jun 29, 2018
Due to utf8mb4 changes, only ezsystems/ezpublish-legacy version compatible is 2018.06, to be released soon. All other versions will fail with the error described in ezsystems/ezpublish-legacy#1361
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants