Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for compression protocol #219

Merged
merged 3 commits into from
Feb 13, 2024

Conversation

mirromutth
Copy link
Contributor

@mirromutth mirromutth commented Feb 5, 2024

Motivation:

See also #216 , MySQL Connect Options --compression-algorithms

Modification:

  • Refactor message codecs, move sequence ID handling to MessageDuplexCodec
  • Add compression codec and support for zstd and zlib
  • Add configuration options for compression algorithms and zstd compression level
  • Refactor the configuration way in the IntegrationTestSupport

I refactored codec handlers, used the original Sequence ID concept instead of "Envelope ID", there are two main reasons for this:

  1. The QueryFlow should not consider how the packet should be encoded, the "envelope ID" is obviously something that the encoding layer needs to consider
    • This problem also occurred when implementing LOCAL INFILE. The QueryFlow needs to constructing LocalInfileResponse with an envelope ID, but this is obviously not any logic about LOCAL INFILE
    • This also causes MessageDecoder to need to additionally consider whether to pass the envelope ID to ServerMessage
  2. Compression protocol contains a compression sequence ID, it reset in the same time as the uncompressed sequence ID, but incremented independently.
    • If we support this like before, we need consider two envelope counter for uncompressed sequence ID and compression sequence ID at QueryFlow, all ClientMessages and all ServerMessages, which is bad.

The reason why this concept was created in the early days was that the relationship of sequence IDs between packets was not complicated at that time, and it was reset to 0 every time in the command phase.

Result:

User can configure the compression algorithm to enable the MySQL compression protocol and the driver will auto select the preferred algorithm to use. The compression protocol is disabled by default.

@mirromutth mirromutth added the enhancement New feature or request label Feb 5, 2024
@mirromutth mirromutth linked an issue Feb 5, 2024 that may be closed by this pull request
@mirromutth mirromutth force-pushed the 216-feature-compression-support branch 2 times, most recently from 14ebe8e to 8967db5 Compare February 5, 2024 07:26
Copy link
Collaborator

@jchrys jchrys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's quite extensive, so it'll take me a bit longer to review thoroughly. Appreciate your patience!

@jchrys jchrys added this to the 1.1.1 milestone Feb 6, 2024
@mirromutth mirromutth force-pushed the 216-feature-compression-support branch from 59f20da to 0b8f17a Compare February 9, 2024 03:06
Copy link
Collaborator

@jchrys jchrys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT, just tiny details.

@jchrys
Copy link
Collaborator

jchrys commented Feb 13, 2024

btw)
could you check below discussions once you have time?
#190
#166
above are related to micronaut's issue below.
micronaut-projects/micronaut-data#2388

- Refactor message codecs, move sequence ID handling to MessageDuplexCodec
- Add compression codec and support for zstd and zlib
- Add configuration options for compression algorithms and zstd compression level
- Refactor the configuration way at `IntegrationTestSupport`
@mirromutth mirromutth force-pushed the 216-feature-compression-support branch from 0b8f17a to 7ad491a Compare February 13, 2024 06:24
Copy link
Collaborator

@jchrys jchrys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mirromutth mirromutth merged commit fcf1f18 into trunk Feb 13, 2024
12 checks passed
@mirromutth mirromutth deleted the 216-feature-compression-support branch February 13, 2024 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature] Compression protocol support
2 participants