Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drainer,pump: Add support for enabling gzip grpc compression #495

Merged
merged 1 commit into from
Mar 27, 2019
Merged

drainer,pump: Add support for enabling gzip grpc compression #495

merged 1 commit into from
Mar 27, 2019

Conversation

suzaku
Copy link
Contributor

@suzaku suzaku commented Mar 22, 2019

What problem does this PR solve?

Add support for enabling gzip compression of payloads between Pump and Drainer Issue

What is changed and how it works?

A new flag compressor is added, which can be "gzip" or "" (compressor disabled).
The value is "" by default, so original users won't get compression turned on by accident after upgrading to a version that include this change.

Check List

Tests

  • Integration test
  • Manual test
    1. stats.Handler is used to verify that the payload is indeed compressed
    2. When compressor is used and the register of gzip is commented out on the server side, the requests are observed to be rejected.

Code changes

Side effects

Related changes

@suzaku
Copy link
Contributor Author

suzaku commented Mar 22, 2019

/run-all-tests tidb=release-2.1 tikv=release-2.1 pd=release-2.1

Copy link
Contributor

@july2993 july2993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

tests/binlog/drainer.toml Outdated Show resolved Hide resolved
@@ -83,6 +85,8 @@ func NewServer(cfg *Config) (*Server, error) {
}

ctx, cancel := context.WithCancel(context.Background())
ctx = context.WithValue(ctx, drainerKeyType("compressor"), cfg.Compressor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be better just pass by add param fro NewPump?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to get this piece of information from cfg in Server and pass it all the way down to Pump.
But it turned out I would also have to add a corresponding field in Collector and Pump because the part that may need to know compressor was not called when these structs were created.
So I guess using context.WithValue might be a little simpler, for example, when I changed enable_compression to compressor I only need to update two or three lines of code.

cmd/drainer/drainer.toml Outdated Show resolved Hide resolved
Copy link
Contributor

@july2993 july2993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@july2993
Copy link
Contributor

@WangXiangUSTC @GregoryIan PTAL

drainer/config.go Outdated Show resolved Hide resolved
@WangXiangUSTC
Copy link
Contributor

rest LGTM

@july2993
Copy link
Contributor

/run-all-tests tidb=release-2.1 tikv=release-2.1 pd=release-2.1

@july2993
Copy link
Contributor

@WangXiangUSTC PTAL

Copy link
Contributor

@WangXiangUSTC WangXiangUSTC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants