Add initial support for delta xDS #882
Conversation
Jake-Shadle
requested review from
markmandel,
XAMPPRocky and
koslib
as code owners
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
Jake-Shadle
force-pushed
the
incremental-xds
branch
from
153a46b to
fd9cd32
Compare
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
2 similar comments
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
If the metadata isn't taken into account during hashing of endpoint sets, server end will think the client has the most up to date version of the set and not send it. Luckily the relay_routing test exposed this issue
Jake-Shadle
force-pushed
the
incremental-xds
branch
from
c4cd6ed to
08a1505
Compare
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
1 similar comment
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
Jake-Shadle
force-pushed
the
incremental-xds
branch
from
2db3caf to
51daaf7
Compare
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
Jake-Shadle
force-pushed
the
incremental-xds
branch
from
51daaf7 to
3ed5b4d
Compare
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
Jake-Shadle
force-pushed
the
incremental-xds
branch
from
3ed5b4d to
f4354b2
Compare
|
This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size. |
|
Build Succeeded 🥳 Build Id: 0a06e60e-2d80-4779-a139-21cdddacba6e The following development images have been built, and will exist for the next 30 days: To build this version: |
markmandel
added
kind/feature
This adds initial support for delta xDS to the proxy, relay, and agent. Both the client and server side keep track of the resource versions (only for the clustermap, the listener/filter chain doesn't do anything different from the state of the world xDS streams since there's only ever 1 and is small) that it currently has, so that reconnections to a new server on the client side can send what clustermap versions it currently has to avoid getting duplicate data sent on initial connection, and update to the clustermap on the server side will only ever send the localities the client side doesn't have up to date versions of, instead of all of them.
The version info is currently tracked in two ways, 1 is a simple incrementing integer in each locality that is only used by Watch to detect if changes occur, but the other one is a full 64-bit hash of every endpoint address and the quilkin.dev/token set in the locality. This is currently done with seahash, though I haven't done a benchmark of it against other potential hash implementations, but that can be revisited later. This hash calculation is done only in agents, and that hash is used as the xDS resource version, so both management and proxy servers just parse and use the hash as is, without recalculating it. Since the hash is calculated with constant seeds this means the hash is stable across different machines for the same input so that resources can be limited to the ones that the client side actually doesn't have.
The current code will attempt to connect to the delta stream first, and fallback to the old state of the world streams if the remote end doesn't support it, the way tonic streams work makes this a bit ugly, but hopefully this can be removed in a future version in favor of only using delta streams.
Resolves: #792