-
-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Increase backpressure threshold to 5 seconds #340
Conversation
During inc-626 we saw excessive backpressure which caused the consumer to be paused and resumed a lot. This led to excessive network traffic as the local queue was purged and messages were re-downloaded. Increase the threshold for pausing to reduce the impact of this.
tests/processing/test_processor.py
Outdated
@@ -86,7 +86,7 @@ def test_stream_processor_lifecycle() -> None: | |||
with assert_changes(lambda: int(consumer.pause.call_count), 0, 1): | |||
processor._run_once() | |||
assert strategy.submit.call_args_list[-1] == mock.call(message) | |||
time.sleep(1) | |||
time.sleep(5) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤮
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please mock time.time
instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
tests/processing/test_processor.py
Outdated
@@ -86,7 +86,7 @@ def test_stream_processor_lifecycle() -> None: | |||
with assert_changes(lambda: int(consumer.pause.call_count), 0, 1): | |||
processor._run_once() | |||
assert strategy.submit.call_args_list[-1] == mock.call(message) | |||
time.sleep(1) | |||
time.sleep(5) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please mock time.time
instead
During inc-626 we saw excessive backpressure which caused the consumer to be paused and resumed a lot. This led to excessive network traffic as the local queue was purged and messages were re-downloaded. Increase the threshold for pausing to reduce the impact of this.