-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CSDS: add new config status ACKED #13121
Changes from all commits
e142072
78c118c
ef87599
8f2fb64
15a4491
abd2351
5fb88f0
87b889f
0ba9254
f7f38af
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,6 +9,7 @@ import "envoy/type/matcher/v3/node.proto"; | |
import "google/api/annotations.proto"; | ||
import "google/protobuf/struct.proto"; | ||
|
||
import "udpa/annotations/migrate.proto"; | ||
import "udpa/annotations/status.proto"; | ||
import "udpa/annotations/versioning.proto"; | ||
|
||
|
@@ -21,9 +22,8 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE; | |
// [#protodoc-title: Client Status Discovery Service (CSDS)] | ||
|
||
// CSDS is Client Status Discovery Service. It can be used to get the status of | ||
// an xDS-compliant client from the management server's point of view. In the | ||
// future, it can potentially be used as an interface to get the current | ||
// state directly from the client. | ||
// an xDS-compliant client from the management server's point of view. It can | ||
// also be used to get the current xDS states directly from the client. | ||
service ClientStatusDiscoveryService { | ||
rpc StreamClientStatus(stream ClientStatusRequest) returns (stream ClientStatusResponse) { | ||
} | ||
|
@@ -34,7 +34,7 @@ service ClientStatusDiscoveryService { | |
} | ||
} | ||
|
||
// Status of a config. | ||
// Status of a config from a management server view. | ||
enum ConfigStatus { | ||
// Status info is not available/unknown. | ||
UNKNOWN = 0; | ||
|
@@ -49,10 +49,30 @@ enum ConfigStatus { | |
// ACK/NACK. | ||
STALE = 3; | ||
|
||
// Management server has sent the config to client but received NACK. | ||
// Management server has sent the config to client but received NACK. The | ||
// attached config dump will be the latest config (the rejected one), since | ||
// it is the persisted version in the management server. | ||
ERROR = 4; | ||
} | ||
|
||
// Config status from a client-side view. | ||
enum ClientConfigStatus { | ||
// Config status is not available/unknown. | ||
CLIENT_UNKNOWN = 0; | ||
|
||
// Client requested the config but hasn't received any config from management | ||
// server yet. | ||
CLIENT_REQUESTED = 1; | ||
|
||
// Client received the config and replied with ACK. | ||
CLIENT_ACKED = 2; | ||
|
||
// Client received the config and replied with NACK. Notably, the attached | ||
// config dump is not the NACKed version, but the most recent accepted one. If | ||
// no config is accepted yet, the attached config dump will be empty. | ||
CLIENT_NACKED = 3; | ||
} | ||
|
||
// Request for client status of clients identified by a list of NodeMatchers. | ||
message ClientStatusRequest { | ||
option (udpa.annotations.versioning).previous_message_type = | ||
|
@@ -67,12 +87,20 @@ message ClientStatusRequest { | |
} | ||
|
||
// Detailed config (per xDS) with status. | ||
// [#next-free-field: 7] | ||
// [#next-free-field: 8] | ||
message PerXdsConfig { | ||
option (udpa.annotations.versioning).previous_message_type = | ||
"envoy.service.status.v2.PerXdsConfig"; | ||
|
||
ConfigStatus status = 1; | ||
// Config status generated by management servers. Will not be present if the | ||
// CSDS server is an xDS client. | ||
ConfigStatus status = 1 [(udpa.annotations.field_migrate).oneof_promotion = "status_config"]; | ||
|
||
// Client config status is populated by xDS clients. Will not be present if | ||
// the CSDS server is an xDS server. No matter what the client config status | ||
// is, xDS clients should always dump the most recent accepted xDS config. | ||
ClientConfigStatus client_status = 7 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you might need to have a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lidi, I think you still need to address this comment. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to the style in the code base, the So, in my mind, keeping There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, that's what I meant. The v4alpha file looks right now. The last time I looked, I thought it was still wrong, but maybe I was looking at an old snapshot. Anyway, this looks great. Thanks! |
||
[(udpa.annotations.field_migrate).oneof_promotion = "status_config"]; | ||
|
||
oneof per_xds_config { | ||
admin.v3.ListenersConfigDump listener_config = 2; | ||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have a similar statement added about control plane behavior for configs with a negative status, i.e. "this is always the latest successful config?" Or maybe that's not even desired, we might want the latest successful and unsuccessful config? Would that also make sense for client? CC @fuqianggao
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This behavior wasn't clearly described in this proto definition before, or in other documentation. Could it be a control-plane specific detail?
Hi @fuqianggao, can you help to explain the designed behavior of config dump when client replied with NACK?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For control plane, it will be the latest config that control plane has. If status is NACK, it means the config is sent to client, but rejected.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. I think the actionable thing is for @lidizheng to update the management server wording to reflect this. I'm still wondering in this discussion thread, is there value in having both active and failed configs reported?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect that for both xDS servers and xDS clients, there would be additional overhead in storing old versions of configs. In general, both sides will store only the latest version that they care about, so they don't have anything else to report here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And if all clients implement CSDS, the CLI can get the active config from the clients, and the most up-to-date config from the management server (if they are different).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have updated the comment for
ConfigStatus.ERROR
to note that the management server will dump the latest config in case of client replying NACK.The behavior described by @menghanl is implemented by
istioctl
'sproxy-status
command.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the idea of being able to query both client and server CSDS and effectively determine the failing and active config. @lidizheng can you please file an issue for Envoy to add support for CSDS? You don't have to do the implementation, but we should track this as an open feature request that we can share with the community.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filed as an issue: #13181. For Envoy, it overlaps with the admin config dump feature. Today, tools like
istioctl
is fetching xDS config from server via CSDS, and from client via admin config dump (HTTP).