Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

etcdserver: process the case of the last WAL record being partially synced to disk #15068

Merged
merged 2 commits into from
Jan 11, 2023
Merged

etcdserver: process the case of the last WAL record being partially synced to disk #15068

merged 2 commits into from
Jan 11, 2023

Conversation

ahrtr
Copy link
Member

@ahrtr ahrtr commented Jan 7, 2023

There is a regression in #14122. When the last record is only partially synced to disk, we should return io.ErrUnexpectedEOF in the error chain, otherwise etcdserver will not repair it and etcdserver will also fail to get started.

See also #14114 (comment)

@ahrtr
process the scenaro of the last WAL record being partially synced to …
bd9f158 
…disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
@ahrtr
remove the dependency on the deprecated io/ioutil
5ef713c 
Reference: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Benjamin Wang <wachao@vmware.com>
@ahrtr ahrtr changed the title Process the case of the last record being partially synced to disk Process the case of the last WAL record being partially synced to disk Jan 7, 2023
ahrtr added a commit to ahrtr/etcd that referenced this pull request Jan 7, 2023
@ahrtr
etcdserver: process the scenaro of the last WAL record being partiall…
e1fc545 
…y synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
@ahrtr ahrtr mentioned this pull request Jan 7, 2023
Merged
@codecov-commenter
Copy link

Codecov Report

Merging #15068 (5ef713c) into main (6200b22) will decrease coverage by 0.25%.
The diff coverage is 37.50%.

@@            Coverage Diff             @@
##             main   #15068      +/-   ##
==========================================
- Coverage   74.87%   74.61%   -0.26%     
==========================================
  Files         415      415              
  Lines       34288    34288              
==========================================
- Hits        25672    25583      -89     
- Misses       6994     7059      +65     
- Partials     1622     1646      +24
Flag Coverage Δ
all 74.61% <37.50%> (-0.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/etcdserver/bootstrap.go 68.70% <0.00%> (ø)
server/storage/wal/wal.go 57.38% <25.00%> (ø)
server/storage/wal/decoder.go 89.00% <100.00%> (+3.00%) ⬆️
client/v3/experimental/recipes/queue.go 58.62% <0.00%> (-6.90%) ⬇️
client/pkg/v3/fileutil/purge.go 68.85% <0.00%> (-6.56%) ⬇️
server/etcdserver/api/rafthttp/msgappv2_codec.go 69.56% <0.00%> (-5.22%) ⬇️
client/v3/concurrency/mutex.go 61.03% <0.00%> (-5.20%) ⬇️
server/storage/mvcc/kvstore_compaction.go 95.65% <0.00%> (-4.35%) ⬇️
server/etcdserver/api/v3rpc/auth.go 79.34% <0.00%> (-4.35%) ⬇️
server/etcdserver/api/v3rpc/watch.go 83.49% <0.00%> (-4.13%) ⬇️
... and 23 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tjungblu
Copy link
Contributor

tjungblu commented Jan 9, 2023

/lgtm (non-binding)

@ahrtr ahrtr changed the title Process the case of the last WAL record being partially synced to disk etcdserve: process the case of the last WAL record being partially synced to disk Jan 11, 2023
@ahrtr ahrtr changed the title etcdserve: process the case of the last WAL record being partially synced to disk etcdserver: process the case of the last WAL record being partially synced to disk Jan 11, 2023
@ahrtr ahrtr requested a review from ptabor January 11, 2023 08:59
@ptabor ptabor changed the title etcdserver: process the case of the last WAL record being partially synced to disk etcdserver: process the case of the last WAL record being partially synced to disk [backport 3.5] Jan 11, 2023
@ahrtr ahrtr merged commit ef917f1 into etcd-io:main Jan 11, 2023
@ptabor ptabor changed the title etcdserver: process the case of the last WAL record being partially synced to disk [backport 3.5] etcdserver: process the case of the last WAL record being partially synced to disk Jan 11, 2023
tjungblu pushed a commit to tjungblu/etcd that referenced this pull request Jan 11, 2023
@ahrtr @tjungblu
UPSTREAM:<carry>: etcdserver: process the scenaro of the last WAL rec…
15f68c3 
…ord being partially synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
@tjungblu tjungblu mentioned this pull request Jan 11, 2023
Merged
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/etcd that referenced this pull request Jan 11, 2023
@ahrtr
UPSTREAM:<carry>: etcdserver: process the scenaro of the last WAL rec…
3e579cb 
…ord being partially synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/etcd that referenced this pull request Jan 16, 2023
@ahrtr
UPSTREAM:<carry>: etcdserver: process the scenaro of the last WAL rec…
c4b33a5 
…ord being partially synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/etcd that referenced this pull request Jan 17, 2023
@ahrtr
UPSTREAM:<carry>: etcdserver: process the scenaro of the last WAL rec…
bd865d7 
…ord being partially synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/etcd that referenced this pull request Jan 23, 2023
@ahrtr
UPSTREAM:<carry>: etcdserver: process the scenaro of the last WAL rec…
a0ccfca 
…ord being partially synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
tjungblu pushed a commit to tjungblu/etcd that referenced this pull request Jul 26, 2023
@ahrtr @tjungblu
etcdserver: process the scenaro of the last WAL record being partiall…
e9edcf4 
…y synced to disk

We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Backport etcd-io#15068

Signed-off-by: Benjamin Wang <wachao@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ptabor ptabor ptabor approved these changes

Assignees
No one assigned
Labels
backport/v3.5
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ahrtr @codecov-commenter @tjungblu @ptabor