Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Servers should be able to join a cluster #2966

Closed
2 of 4 tasks
pauldix opened this issue Jun 12, 2015 · 4 comments
Closed
2 of 4 tasks

Servers should be able to join a cluster #2966

pauldix opened this issue Jun 12, 2015 · 4 comments
Assignees
Milestone

Comments

@pauldix
Copy link
Member

pauldix commented Jun 12, 2015

Servers should be able to join a cluster. Here's a rough outline of how it should work, but please ask questions.

When a server starts up, if it doesn't have a metastore (on disk), it should do one of the following:

  • If join was given as a command line argument, it should attempt to join the cluster at that URL
  • If no join was specified, it should create a new cluster of just itself
  • if join was specified, but they have the metastore on disk, ignore the join

Join

Here's how join should work. The URL supplied should be a server (or a load balancer that points to many InfluxDB servers). The point is that it should be able to join a cluster when pointed to any server in that cluster.

The first 3 servers to join the cluster should run the meta service (Raft). Any servers that join after that should just get a copy of the metastore. They should keep the cached version on disk.

TODO

  • Incremental joins
  • Caching metastore
  • Promoting server to run consensus service
  • Demoting server so it drops out of the consensus group and stops running the service
@pauldix pauldix added this to the 0.9.1 milestone Jun 12, 2015
@btashton
Copy link
Contributor

I really wonder if you are reinventing the wheel here. What advantage does this give over something like etcd and a discovery URL? And if you dont want to use etcd, perhaps it is a good model. I know I was looking at using it to generate the peers list for the config file.

jwilder added a commit that referenced this issue Jun 25, 2015
@toddboom toddboom modified the milestones: 0.9.1, 0.9.2 Jul 2, 2015
@pauldix
Copy link
Member Author

pauldix commented Jul 3, 2015

@btashton that's the role that Raft is playing for us. It's just integrated into the InfluxDB cluster. What this gives us is a simpler deployment story and less things to manage.

Yes, it's reinventing the wheel, but that's what incremental progress in software is all about. Improving things just a little bit along the way. And this gives us an improvement in usability for our users.

@beckettsean beckettsean modified the milestones: 0.9.3, 0.9.2 Jul 8, 2015
@rynbrd
Copy link

rynbrd commented Jul 15, 2015

Based on the description it sounds possible that this would allow a new cluster to be deployed automatically in AWS via an AutoScaling group and ELB. The tricky part is when the first node comes online and is added to the ELB. If passed a cluster URL pointing to the ELB it will have no available backends to connect to. As long as the node creates a new cluster from itself when it's unable to contact the cluster or if the only node in the cluster is itself then things will work out.

We do something similar with our CoreOS/etcd clusters in AWS, though they require a discovery service be running.

@beckettsean
Copy link
Contributor

@jwilder can we close this?

@jwilder jwilder closed this as completed Aug 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants