Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling unsupported compression codec (#795) #798

Merged
merged 3 commits into from
Nov 22, 2021
Merged

Conversation

ods
Copy link
Collaborator

@ods ods commented Nov 20, 2021

Changes

Fixes #795

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • Add a new news fragment into the CHANGES folder

@codecov
Copy link

codecov bot commented Nov 20, 2021

Codecov Report

Merging #798 (0a5b0ba) into master (fcee7d1) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #798      +/-   ##
==========================================
- Coverage   97.89%   97.87%   -0.03%     
==========================================
  Files          31       31              
  Lines        5279     5307      +28     
==========================================
+ Hits         5168     5194      +26     
- Misses        111      113       +2     
Flag Coverage Δ
cext 88.58% <25.00%> (-0.36%) ⬇️
integration 97.83% <100.00%> (-0.03%) ⬇️
purepy 97.39% <100.00%> (-0.03%) ⬇️
unit 51.89% <100.00%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aiokafka/errors.py 100.00% <ø> (ø)
aiokafka/record/default_records.py 96.01% <100.00%> (+0.15%) ⬆️
aiokafka/record/legacy_records.py 98.97% <100.00%> (+0.05%) ⬆️
aiokafka/consumer/fetcher.py 96.87% <0.00%> (-0.32%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fcee7d1...0a5b0ba. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Nov 20, 2021

This pull request introduces 2 alerts when merging 6483a3d into fcee7d1 - view on LGTM.com

new alerts:

  • 2 for Module is imported with 'import' and 'import from'

@lgtm-com
Copy link

lgtm-com bot commented Nov 20, 2021

This pull request introduces 2 alerts when merging 0a5b0ba into fcee7d1 - view on LGTM.com

new alerts:

  • 2 for Module is imported with 'import' and 'import from'

@@ -73,6 +76,20 @@ class LegacyRecordBase:
LOG_APPEND_TIME = 1
CREATE_TIME = 0

def _assert_has_codec(self, compression_type):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reuse the same function to avoid copy-paste?

Copy link
Collaborator Author

@ods ods Nov 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. What module is the best place to put in?

Copy link
Collaborator Author

@ods ods Nov 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry. They will diverge after adding zstd support. I think that's the reason why it's duplicated in kafka-python: legacy record can't be used with zstd compression. But I didn't verified this, so the difference may be a mistake too.

Copy link
Collaborator Author

@ods ods Nov 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to KIP-110

Zstd will only be allowed with magic = 2 format

Copy link
Member

@tvoinarovskyi tvoinarovskyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, let's see how it looks with zstd I guess.

@ods ods merged commit 444a1a7 into aio-libs:master Nov 22, 2021
@ods ods deleted the fix795 branch November 22, 2021 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aiokafka some time fails, possibly because no support of ZSTD compression
2 participants