Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NRG: Don't delete RAFT state if stream/consumer creation failed during shutdown #6061

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

MauriceVanVeen
Copy link
Member

Antithesis found a case where if a server was hard killed and would replay RAFT append entries during startup, if it would then be SIGTERM-ed while the RAFT recovery is running, the acc.lookupStream that happens in js.processClusterCreateStream resulted in a JetStream not enabled for account error. Which would then delete the RAFT state.

There are also other errors that could pop up, so either way; if we're shutting down, don't delete the RAFT state.

Signed-off-by: Maurice van Veen github@mauricevanveen.com

@MauriceVanVeen MauriceVanVeen marked this pull request as ready for review October 31, 2024 10:50
@MauriceVanVeen MauriceVanVeen requested a review from a team as a code owner October 31, 2024 10:50
@MauriceVanVeen MauriceVanVeen force-pushed the maurice/dont-delete-raft-state-during-shutdown branch from a888677 to 5d5a329 Compare October 31, 2024 11:05
@MauriceVanVeen MauriceVanVeen changed the title NRG: Don't delete RAFT state if stream creation failed during shutdown NRG: Don't delete RAFT state if stream/consumer creation failed during shutdown Oct 31, 2024
…g shutdown

Signed-off-by: Maurice van Veen <github@mauricevanveen.com>
@MauriceVanVeen MauriceVanVeen force-pushed the maurice/dont-delete-raft-state-during-shutdown branch from 5d5a329 to 6d1ae91 Compare October 31, 2024 11:17
Copy link
Member

@neilalexander neilalexander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@derekcollison derekcollison merged commit 185a6d3 into main Oct 31, 2024
5 checks passed
@derekcollison derekcollison deleted the maurice/dont-delete-raft-state-during-shutdown branch October 31, 2024 15:40
neilalexander added a commit that referenced this pull request Nov 12, 2024
Includes the following:

* Some tweaks to the NRG test helpers
* #6055
* #6061
* #6065 
* #6041 (but with `math/rand`
instead of `math/rand/v2` due to an older Go version in CI for 2.10.x)
* #6066
* #6067
* #6069
* #6075
* #6082
* #6087
* #6086
* #6088
* #6089
* #6092
* #6096
* #6098
* #6097
* #6105
* #6104
* #6106
* #6109
* #6111
* #6112

Signed-off-by: Neil Twigg <neil@nats.io>
neilalexander added a commit that referenced this pull request Nov 25, 2024
Includes the following:

- #5661
- #5666
- #5671
- #5344
- #5684
- #5689
- #5691
- #5714
- #5717
- #5707
- #5792
- #5912
- #5957
- #5700
- #5975
- #5991
- #5987
- #6027
- #6038
- #6053
- #5848
- #6055
- #6056
- #6060
- #6061
- #6072
- #5832
- #6073
- #6107

Signed-off-by: Neil Twigg <neil@nats.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants