Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copier service #3975

Merged
merged 1 commit into from
Sep 3, 2015
Merged

Copier service #3975

merged 1 commit into from
Sep 3, 2015

Conversation

benbjohnson
Copy link
Contributor

Overview

This pull request adds the copier service which allows one server to copy shards from another server. This will be used for moving shards in the cluster.

Additional Notes

  • Utilitizes the existing multiplex'd TCP connection on the server.
  • Protobuf headers are used for request & response to allow for future growth.
  • Response body sent as <uint64(length)><bytes...>
  • copier.Client provided for easier integration.
  • io.WriterTo added to the tsdb.Engine interface and to b1 & bz1 implementations.

@otoolep
Copy link
Contributor

otoolep commented Sep 3, 2015

Seems reasonable, nothing jumped out at me. +1

// readResponse reads and unmarshals a Response from r.
func (c *Client) readResponse(r io.Reader) (*internal.Response, error) {
// Read response length.
var n uint32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a uint64? Shards files can be larger than 4gb, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's just for the response header. The shard writes out its data size as a uint64.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. Maybe this is just the length of the protobuf response.

This commit adds the copier service which allows one server to
copy shards from another server. This will be used for moving
shards in the cluster.
benbjohnson added a commit that referenced this pull request Sep 3, 2015
@benbjohnson benbjohnson merged commit 0163945 into influxdata:master Sep 3, 2015
@benbjohnson benbjohnson deleted the copy-shard branch September 3, 2015 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants