fix(gzip): Make sure Vary header is always added if a response can be compressed

[mattmess1221]

Fixes #2863

Summary

Since the presense or absense of the Accept-Encoding header can change the response body, Vary should always be added whether the client requested gzip encoding or not. Otherwise some caching implementations might return a non-compressed response even when a compressed one was requested.

This change may seem a bit complex, but it reworks the GZipMiddleware to behave the same no matter what the compression is (or lack of). Basic changes include:

1. Move all gzip compression to a method apply_compression and don't update the content-encoding or content-length headers if the body didn't change.
2. Create a base class IdentityResponder from GZipResponder which will be used to respond to non-gzip requests. Its apply_compression method will return the body unchanged.
3. Coincidentally, it's now easier to implement new compression algorithms like deflate, brotli, or zstd. This PR does not implement those.

Checklist

• I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.

[Kludex]

couldn't we just move the add_vary_header up?

[mattmess1221]

Unfortunately, it also needs to be added when the accept-encoding header is not gzip.

[Kludex]

Unfortunately, it also needs to be added when the accept-encoding header is not gzip.

I understand that, that's the goal of this PR. Can't we really avoid this huge diff just to move that up?

[mattmess1221]

My thinking was I would need to implement a responder for non-gzip encodings, and I also wanted to use the same logic for determining if I shouldn't add the Vary header (body is too small, or streaming response with content-encoding already set).

Anyway, I'll try to get it smaller.

[Kludex]

hold, I'm going to review carefully.

No need to try anything for now. Thanks for the PR.

[Kludex]

This PR is actually changing the behavior on when we do the compression. It actually implements what is proposed here: #2753

[mattmess1221]

Without flushing, the tests would fail.

[Kludex]

Without flushing, the tests would fail.

That's a change in behavior. I don't think it should flush on every chunk.

[Kludex]

Thanks. 🙏