Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proper support for collations #135

Merged
merged 5 commits into from
May 15, 2024

Conversation

altmannmarcelo
Copy link
Contributor

The motivation behind this work is with regards to handling CHAR(x) types.
MySQL has support for multi byte collations, such as utf8mb4 (4 bytes). When handling text fields, specially CHAR we should fill the column width by padding spaces to the right.
When retrieving data from binary log for example, we can extract via TME metadata from the columns, such as column size and ID of collation. The problem relies of the fact that column size is expressed in bytes taking into consideration original collation, for example, for a CHAR(2) column using UTF8MB4_0900_AI_CI the column size will be 8.

Here is one example of TME metadata and BinlogValue::Value for above example for a CHAR(2) insert of value 'A'

get_column_type: MYSQL_TYPE_STRING
metadata: [254, 8] //[0] - Real Column Type, [1] - Column size
charset: 255
value: 'A'

Currently we lack proper collation dictionary to identify Column size as character length.

Added required ENUM and struct to lookup proprieties for collations.
Removed UTF8 and UTF8MB4 collation from the constants module and used
the collation module instead.
@blackbeam blackbeam merged commit 2d9e6bc into blackbeam:master May 15, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants