Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A33: Client-Side Fault Injection #201

Merged
merged 13 commits into from
Feb 12, 2021
Merged

A33: Client-Side Fault Injection #201

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b9b6e47
Add gRPC A33: Client-Side Fault Injection
lidizheng Aug 13, 2020
443cb4a
Add grpc-io link
lidizheng Aug 13, 2020
c242ad7
Clearify we only want to support the traffic matcher based on headers
lidizheng Feb 3, 2021
fd9aac0
Refer to A39 for HTTP filter processing
lidizheng Feb 3, 2021
46c5119
Apply review suggestions
lidizheng Feb 4, 2021
afbe719
Apply reviewer's suggestions
lidizheng Feb 5, 2021
81acec2
Some minor polish after read it on GitHub
lidizheng Feb 5, 2021
8b547a4
Adopt reviewer's advices
lidizheng Feb 10, 2021
46d7dfa
Add an environment variable guard section
lidizheng Feb 10, 2021
1f9e4ca
Move the env guard section up
lidizheng Feb 10, 2021
ec1e7cb
Clarify the env guard is temporary
lidizheng Feb 10, 2021
ae03801
Add the description of header percentage cap feature
lidizheng Feb 12, 2021
1382c84
Change the status to Approved
lidizheng Feb 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions A33-Fault-Injection.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,11 @@ gRPC should accept fault injection settings with either HTTP status code or gRPC
According to the [Envoy implementation](https://github.com/envoyproxy/envoy/blob/main/source/extensions/filters/http/fault/fault_filter.cc#L222), each feature of fault injection is checked with a different random number. Compared to using a single random number for all features, this affects the probability of an RPC being affected by fault injection. For example, the fault injection has 20% chance to delay and 5% chance to abort. Under the single random number solution, only 20% RPC will be affected. But if two features are independent, 24% RPC will be affected. So, **this doc suggests to make each feature independent by using a different random number each time.**


### Environment Variable Protection
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make changes similar to what I did in af7b012 to reflect the fact that this is just temporary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Using the same words as your commit.


`GRPC_XDS_EXPERIMENTAL_FAULT_INJECTION` as an environment variable will be used to guard this feature for the initial release. Once it is set, gRPC will start to interpret xDS HTTP filters ([A39: xDS HTTP Filters](https://github.com/grpc/proposal/pull/219)) and enforce fault injection if configured.


## Alternatives

### Alternative 1: Fault Injection Config via ProtoBuf Metadata
Expand Down Expand Up @@ -193,11 +198,6 @@ Envoy uses the token bucket algorithm to throttle the speed of letting response
Envoy implements fault injection is an `HTTPFilter`, which can choose to operate on bytes or HTTP messages or events. It is unclear if gRPC can perform the precise same behavior, since gRPC generally operates on proto messages (only Core has visibility to bytes, not Java/Golang). More importantly, there is not enough consensus for if the timeout timer should start at the beginning of an RPC, or the receipt of the first bytes from peer, or after the entire message is received.


### Environment variable protection

`GRPC_XDS_EXPERIMENTAL_FAULT_INJECTION` as an environment variable will be used to guard this feature for the initial release. Once it is set, gRPC will start to interpret xDS HTTP filters ([A39: xDS HTTP Filters](https://github.com/grpc/proposal/pull/219)) and enforce fault injection if configured.


## Implementation

gRPC Core: https://github.com/grpc/grpc/pull/24354