-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc: v3api rfc #2675
Merged
+463
−0
Merged
doc: v3api rfc #2675
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,191 @@ | ||
## Design | ||
|
||
1. Flatten binary key-value space | ||
|
||
2. Keep the event history until compaction | ||
- access to old version of keys | ||
- user controlled history compaction | ||
|
||
3. Support range query | ||
- Pagination support with limit argument | ||
- Support consistency guarantee across multiple range queries | ||
|
||
4. Replace TTL key with Lease | ||
- more efficient/ low cost keep alive | ||
- a logical group of TTL keys | ||
|
||
5. Replace CAS/CAD with multi-object Tnx | ||
- MUCH MORE powerful and flexible | ||
|
||
6. Support efficient watching with multiple ranges | ||
|
||
7. RPC API supports the completed set of APIs. | ||
- more efficient than JSON/HTTP | ||
- additional tnx/lease support | ||
|
||
8. HTTP API supports a subset of APIs. | ||
- easy for people to try out etcd | ||
- easy for people to write simple etcd application | ||
|
||
|
||
## Protobuf Defined API | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Split into a |
||
|
||
[protobuf](./v3api.proto) | ||
|
||
### Examples | ||
|
||
#### Put a key (foo=bar) | ||
``` | ||
// A put is always successful | ||
Put( PutRequest { key = foo, value = bar } ) | ||
|
||
PutResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 1, | ||
raft_term = 0x1, | ||
} | ||
``` | ||
|
||
#### Get a key (assume we have foo=bar) | ||
``` | ||
Get ( RangeRequest { key = foo } ) | ||
|
||
RangeResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 1, | ||
raft_term = 0x1, | ||
kvs = { | ||
{ | ||
key = foo, | ||
value = bar, | ||
create_index = 1, | ||
mod_index = 1, | ||
version = 1; | ||
}, | ||
}, | ||
} | ||
``` | ||
|
||
#### Range over a key space (assume we have foo0=bar0… foo100=bar100) | ||
``` | ||
Range ( RangeRequest { key = foo, end_key = foo80, limit = 30 } ) | ||
|
||
RangeResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 100, | ||
raft_term = 0x1, | ||
kvs = { | ||
{ | ||
key = foo0, | ||
value = bar0, | ||
create_index = 1, | ||
mod_index = 1, | ||
version = 1; | ||
}, | ||
..., | ||
{ | ||
key = foo30, | ||
value = bar30, | ||
create_index = 30, | ||
mod_index = 30, | ||
version = 1; | ||
}, | ||
}, | ||
} | ||
``` | ||
|
||
#### Finish a tnx (assume we have foo0=bar0, foo1=bar1) | ||
``` | ||
Tnx(TnxRequest { | ||
// mod_index of foo0 is equal to 1, mod_index of foo1 is greater than 1 | ||
compare = { | ||
{compareType = equal, key = foo0, mod_index = 1}, | ||
{compareType = greater, key = foo1, mod_index = 1}} | ||
}, | ||
// if the comparison succeeds, put foo2 = bar2 | ||
success = {PutRequest { key = foo2, value = success }}, | ||
// if the comparison fails, put foo2=fail | ||
failure = {PutRequest { key = foo2, value = failure }}, | ||
) | ||
|
||
TnxResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 3, | ||
raft_term = 0x1, | ||
succeeded = true, | ||
responses = { | ||
// response of PUT foo2=success | ||
{ | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 3, | ||
raft_term = 0x1, | ||
} | ||
} | ||
} | ||
``` | ||
|
||
#### Watch on a key/range | ||
|
||
``` | ||
Watch( WatchRequest{ | ||
key = foo, | ||
end_key = fop, // prefix foo | ||
start_index = 20, | ||
end_index = 10000, | ||
// server decided notification frequency | ||
progress_notification = true, | ||
} | ||
… // this can be a watch request stream | ||
) | ||
|
||
// put (foo0=bar0) event at 3 | ||
WatchResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 3, | ||
raft_term = 0x1, | ||
event_type = put, | ||
kv = { | ||
key = foo0, | ||
value = bar0, | ||
create_index = 1, | ||
mod_index = 1, | ||
version = 1; | ||
}, | ||
} | ||
… | ||
|
||
// a notification at 2000 | ||
WatchResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 2000, | ||
raft_term = 0x1, | ||
// nil event as notification | ||
} | ||
|
||
… | ||
|
||
// put (foo0=bar3000) event at 3000 | ||
WatchResponse { | ||
cluster_id = 0x1000, | ||
member_id = 0x1, | ||
index = 3000, | ||
raft_term = 0x1, | ||
event_type = put, | ||
kv = { | ||
key = foo0, | ||
value = bar3000, | ||
create_index = 1, | ||
mod_index = 3000, | ||
version = 2; | ||
}, | ||
} | ||
… | ||
|
||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm curious about this - what will the effective difference be with this change? Will this change allow bulk loading (
PUT
's)? If so, what will the data format need to be?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes (with empty compare list). Take a look around Line 191 for the format.
We might limit the total size of a bulk loading.