Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to dynamically change cluster membership #81

Open
btmorr opened this issue Jul 3, 2020 · 0 comments
Open

Add ability to dynamically change cluster membership #81

btmorr opened this issue Jul 3, 2020 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@btmorr
Copy link
Owner

btmorr commented Jul 3, 2020

See Section 6 of the extended Raft paper (https://raft.github.io/raft.pdf)

In order to add or replace nodes from a cluster, members have to be able to agree about the new roster and transition to it gracefully. Roughly, the tasks involved are:

  • Create ClusterConfig and JointConfig log record type (as differentiated from current log records)
  • Change the type of Log Entries to a union of LogRecord, ClusterConfig, and JointConfig
  • Add an endpoint or similar mechanism that an administrator can use to initiate a configuration change (such as POST /configure with a body that includes the new membership)
  • On start, a node relies on environment variables to determine valid cluster membership, but as soon as a leader is elected it should create a ClusterConfig that matches the configuration from the environment variable and commit it to the log
  • When a change in cluster membership is initiated, the leader uses the requested new membership to create a JointConfig with an "old" field describing the current configuration, and a "new" field describing the requested change. This is added to the log and sent out for append and then commit as with regular LogRecords, except that it must be appended by a majority of both the old and combined membership before being committed
  • As soon as the JointConfig is committed, the leader may return a success response to the admin endpoint, since at this point even if the leader fails the new configuration will succeed
  • During the JointConfig period, any LogAppend requests have the same acceptance requirement as the JointConfig (majority of both the old and combined membership)
  • As soon as possible after a JointConfig is appended, the leader should create a ClusterConfig entry with the new membership only, append and send it, and then commit it (this time only requiring the members of the new configuration). After this point nodes that are not members of the new configuration no longer participate.

Note:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants