-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UDP Client: Split large points #7305
Conversation
Addr string | ||
|
||
// PayloadSize is the maximum size of a UDP client message, optional | ||
// Tune this based on your network. Defaults to UDPBufferSize. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is UDPBufferSize
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A typo that should be updated to UDPPayloadSize
:-)
8a28850
to
84c5d52
Compare
My latest commit adds some improvements to the For now, these are only taken advantage of in the UDP client, but could be used by the HTTP client as well. I also removed the |
de29fa0
to
2497728
Compare
@phemmer @vlasad @maksadbek @sebito91 You've all been involved with this issue for a while now. I think this current change is good enough to be merged. I would really appreciate any testing you could do with your data that would stress the point splitting behavior while ensuring correctness. I've added some changes that should increase performance over previous iterations: mostly memory re-use and in-place scanning, rather than the decode-reencode process that was happening previously. I still need to update the docs for the client to describe the splitting behavior, along with the reasoning behind the various decisions we've come to over the past 2.5 months :-) |
@joelegasse thank you sir, we'll test this out now and get back to you ASAP. |
@sebito91 Did you get a chance to test this out? |
The v2 UDP client will attempt to split points that exceed the configured payload size. It will only do this for points that have a timestamp specified.
This change also updates the UDP client to take advantage of these improvements, as well as some code review changes.
fb0004b
to
879378e
Compare
The UDP client now supports splitting single points that exceed the configured | ||
payload size. The logic for doing so is listed here, starting with an empty payload. | ||
|
||
1. If a point would cause the current payload to exceed the configured size, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/would/would not/?
879378e
to
2cbd411
Compare
Apologies fr the delay, but this works well for us! |
The v2 UDP client will attempt to split points that exceed the configured payload size. It will only do this for points that have a timestamp specified.
This follows from several ancestors: #6913 #7288
Required for all non-trivial PRs
Required only if applicable
You can erase any checkboxes below this note if they are not applicable to your Pull Request.