fix(worker_event): send large payload #11117
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
to be sent in events lib. In some cases, Kong needs to send big payload. For example dao:crud rely on events to invalidate cache. In these cases, the size of the payload may exceeds the limit due to a large dao entity. In this PR, the `post` method of the worker_events is overwritten to take a retry action when we get the 'payload too big' error message, and the retry action is just to re-post the event with a truncated payload so that the subsequent logic that relies on the event could have a chance to be informed rather than unware at all. Based on this purpose, the retry action is taken and only takn once when the error happens. FTI-4963 KAG-1523
Co-authored-by: Chrono <chrono_cpp@me.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
using worker_event lib. This limit prevent `too big` payload from being published through worker_event. This PR provide a solution for this case. The payload will be sent in serval smaller frame when the payload exceeds the limit, and they will be assembled at the side of the receiver. FTI-4963
|
It seems to be a complex solution? should we increase the limitation of lua-resty-events? |
liverpool8056
force-pushed
the
feature/event-payload-too-big-re-assembly
branch
from
1da96fe
to
f81fafa
Compare
|
@chronolaw glad to see that, so set this as a draft |
|
PR Kong/lua-resty-events#37 increase the limitatioin to 16MB, do you think it is enough? |
chronolaw
added
the
pr/discussion
3 tasks
|
PR #11203 will replace this PR, waiting the final decision. |
|
We made a decision to add a new parameter for specifying the limit in worker_events, referring to Kong/lua-resty-events#37, this PR can be closed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
There is a limit for the size of the payload when using worker_event lib. This limit prevent
too bigpayload from being published through worker_event. This PR provide a solution for this case. The payload will be sent in serval smaller frame when the payload exceeds the limit, and they will be assembled at the side of the receiver.Checklist
Full changelog
Issue reference
FTI-4963