This repository has been archived by the owner on Jun 19, 2022. It is now read-only.
Retry pod memory usage grows without bound #1102
Labels
area/broker
kind/bug
Something isn't working
priority/1
Blocks current release defined by release/* label or blocks current milestone
release/1
storypoint/3
Milestone
Describe the bug
As mentioned in #876, when the retry service has consumers with long timeouts, its memory usage grows until evicted.
Expected behavior
Retry service memory usage remains stable as load increases.
To Reproduce
#876 (comment)
Additional context
This is likely caused by max outstanding bytes being set too high. With hundreds of triggers each consuming messages slower than they're coming in, eventually the retry service will use max outstanding bytes * number of triggers. We can reduce the max outstanding bytes/messages as suggested in #876.
The text was updated successfully, but these errors were encountered: