Refactor raft state handling #1615
Conversation
This commit changes the state handling of the raft log. The actions for related to each raft state are strictly confined within that state's loop. To transition between states, the raft log now much clean up all its actions before moving on. This fixes issues where goroutines were kicked off for one state but were delayed in their scheduling so they would begin after the log had already changed to another state.
| // AfterElectionTimeout returns a channel that fires after the election timeout. | ||
| func (c *Clock) AfterElectionTimeout() <-chan chan struct{} { return newClockChan(c.ElectionTimeout) } | ||
| // AfterElectionTimeout returns a channel that fires after a duration that is | ||
| // between the election timeout and double the election timeout. |
There was a problem hiding this comment.
This matches the Raft paper. If all the nodes return the same timeout then they tend to get stuck in an election loop indefinitely where they can't get enough votes.
|
This is a big change, difficult to grok it all. I'll take another pass after feedback and a green build. |
benbjohnson
changed the title
| l.term = hb.term | ||
| l.votedFor = 0 | ||
| } | ||
| if hb.commitIndex > l.commitIndex { |
There was a problem hiding this comment.
Just so I am sure, here the index of the log is push forward to the index received over the heartbeat. I presume the entries corresponding to the greater commitIndex are also inserted into the log.
There was a problem hiding this comment.
readFromLeader does it, I guess.
There was a problem hiding this comment.
A follower can get a commit index higher than it's uncommitted index if the replication lags behind. That's ok though. It'll just commit those entries when they come in.
|
OK, took a second pass. Still a big change. :-) I do have some questions outstanding about the previous comments, I would like to get some feedback on those. I can follow at a very high-level what is going on, and the tests seem better. If you're happy I think you can merge. |
Refactor raft state handling
Overview
This commit changes the state handling of the raft log. The actions for related to each raft state are strictly confined within that state's loop. To transition between states, the raft log now much clean up all its actions before moving on.
This fixes issues where goroutines were kicked off for one state but were delayed in their scheduling so they would begin after the log had already changed to another state.