-
Notifications
You must be signed in to change notification settings - Fork 239
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A5: Proposal for encoding grpclb data in DNS. #10
Merged
Merged
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
4a25ec6
Proposal for encoding grpclb data in DNS.
markdroth 51c6047
Add link to discussion thread.
markdroth 0c4cb14
Add note about initially not supporting this for Windows or Node.
markdroth cde3d86
Add paragraph about priority and weight fields.
markdroth 067cb33
Merge remote-tracking branch 'upstream/master' into grpclb_in_dns
markdroth 0999e72
Renamed as per https://github.com/grpc/proposal/pull/17
markdroth 7dcb6d0
Added java implementation plan.
markdroth a06f981
Add go implementation information.
markdroth File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,173 @@ | ||
Load Balancing and DNS | ||
---- | ||
* Author(s): Mark D. Roth (roth@google.com) | ||
* Approver: a11r | ||
* Status: Draft | ||
* Implemented in: | ||
* Last updated: 2017-05-01 | ||
* Discussion at: https://groups.google.com/d/topic/grpc-io/6be1QsHyZkk/discussion | ||
|
||
## Abstract | ||
|
||
This document describes how additional name service data for load | ||
balancing should be encoded in DNS. | ||
|
||
## Background | ||
|
||
As described in the [name | ||
resolution](https://github.com/grpc/grpc/blob/master/doc/naming.md) doc, | ||
a resolver implementation is expected to return two additional pieces of | ||
data with each address, both related to [load | ||
balancing](https://github.com/grpc/grpc/blob/master/doc/load-balancing.md). | ||
Those two pieces of data are: | ||
|
||
- A boolean indicating whether the address is a backend address (i.e., | ||
the address to use to contact the server directly) or a balancer | ||
address. | ||
|
||
- The name of the balancer, if the address is a balancer address. | ||
This will be used to perform peer authorization. | ||
|
||
This document describes how those two pieces of data should be encoded | ||
when the name service is DNS. | ||
|
||
### Related Proposals: | ||
|
||
N/A | ||
|
||
## Proposal | ||
|
||
This proposal makes use of DNS SRV records, as described in | ||
[RFC-2782](https://tools.ietf.org/html/rfc2782). The SRV record for | ||
grpclb will use the following values: | ||
|
||
- service name: `grpclb` | ||
- protocol: `tcp` | ||
- priority: this will be `0` for all grpclb SRV records | ||
- weight: this will be `0` for all grpclb SRV records | ||
|
||
When the gRPC client library opens a channel for a given server name, | ||
it will ask DNS for these SRV records in addition to the normal address | ||
records. For each SRV record that comes back in the response, the | ||
client library will then do a DNS lookup for address records for the name | ||
specified by the SRV record. It will then return each of the resulting | ||
addresses with the extra fields indicating that the addresses are | ||
balancer addresses and the name from the SRV record. | ||
|
||
Note that the gRPC library will ignore the values of the `priority` and | ||
`weight` fields. Instead, it will put all balancer addresses into a | ||
single list (in whatever order the DNS server returns them) and use the | ||
`pick_first` load-balancing policy to decide which one of them to use. | ||
However, in order to leave open the possibility of adding support for | ||
priority and weight in the future, we recommend that these fields be set | ||
to 0. That way, if we do add support for these fields in the future, | ||
existing records will not break. | ||
|
||
### Example | ||
|
||
For example, let's say that we have the following DNS zone file: | ||
|
||
``` | ||
$ORIGIN example.com. | ||
; grpclb for server.example.com goes to lb.example.com:1234 | ||
_grpclb._tcp.server IN SRV 0 0 1234 lb | ||
; lb.example.com has 3 IP addresses | ||
lb IN A 10.0.0.1 | ||
IN A 10.0.0.2 | ||
IN A 10.0.0.3 | ||
``` | ||
|
||
With this data, a gRPC client will resolve the name `server.example.com` | ||
to the following addresses: | ||
|
||
``` | ||
address=10.0.0.1:1234, is_balancer=true, balancer_name=lb.example.com | ||
address=10.0.0.2:1234, is_balancer=true, balancer_name=lb.example.com | ||
address=10.0.0.3:1234, is_balancer=true, balancer_name=lb.example.com | ||
``` | ||
|
||
### Future Work: Providing Both Server and Balancer Results | ||
|
||
For redundancy, it may be desirable to return both server and balancer | ||
addresses for the same server name. In principle, this will allow the | ||
client to fall back to directly contacting the servers if the load | ||
balancers are unreachable. However, that fallback functionality is not | ||
yet implemented, so it will be the subject of future work. | ||
|
||
However, it is still possible for service owners to set up their DNS | ||
zone files to publish both types of addresses, so that they are prepared | ||
for an eventual future when this fallback functionality has been | ||
implemented. For example: | ||
|
||
``` | ||
$ORIGIN example.com. | ||
server IN A 10.0.0.11 | ||
IN A 10.0.0.12 | ||
; grpclb for server.example.com goes to lb.example.com:1234 | ||
_grpclb._tcp.server IN SRV 0 0 1234 lb | ||
; lb.example.com has 3 IP addresses | ||
lb IN A 10.0.0.1 | ||
IN A 10.0.0.2 | ||
IN A 10.0.0.3 | ||
``` | ||
|
||
For now, if both address records and SRV records are present for the | ||
same server name, the gRPC client will ignore the address records and | ||
return only the SRV records -- in other words, the result for a lookup | ||
of `server.example.com` will be exactly the same in this case as it | ||
would be in the previous example. | ||
|
||
In the future, when we do implement support for this kind of fallback, | ||
this lookup will result in the following addresses: | ||
|
||
``` | ||
address=10.0.0.11:443, is_balancer=false, balancer_name=<unset> | ||
address=10.0.0.12:443, is_balancer=false, balancer_name=<unset> | ||
address=10.0.0.1:1234, is_balancer=true, balancer_name=lb.example.com | ||
address=10.0.0.2:1234, is_balancer=true, balancer_name=lb.example.com | ||
address=10.0.0.3:1234, is_balancer=true, balancer_name=lb.example.com | ||
``` | ||
|
||
Note that port 443 is the default if not specified by the server name | ||
passed to the client library by the application. If the server name | ||
passed in by the application does specify a port, that would be used for | ||
the resulting server addresses, but it would not affect the port used | ||
for the balancer addresses. | ||
|
||
## Rationale | ||
|
||
We considered one alternative, which was adding additional fields to a | ||
TXT record instead of using an SRV record. However, the SRV record | ||
seems to be a much better fit for this purpose. | ||
|
||
## Implementation | ||
|
||
### C-core | ||
|
||
In C-core, this implementation will rely on use of the c-ares DNS | ||
library, which is being added in | ||
[grpc/grpc#7771](https://github.com/grpc/grpc/pull/7771). | ||
|
||
Specifically, we will change the c-ares resolver implementation to | ||
automatically look up the grpclb SRV records for the specified name. If | ||
found, it will then look up the name(s) that the SRV records point to | ||
and return those as balancer addresses. | ||
|
||
Note that, due to platform support issues, we will initially *not* | ||
support the c-ares resolver under Windows or for Node. Alternatives | ||
will need to be found for these environments. | ||
|
||
### Java | ||
|
||
Java will depend on JNDI, Netty DNS, dnsjava, or a another DNS library | ||
to do SRV record resolution. The existing name resolver, `DnsNameResolver`, | ||
will be modified to resolve the additional records and include them in | ||
the Attributes presented to the load balancer. | ||
|
||
### Go | ||
|
||
TODO(roth): Talk to Go folks about content for this section. | ||
|
||
## Open issues (if applicable) | ||
|
||
N/A |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go will use the LookupSRV function in package net to do SRV record resolution. We are going to implement a DNS name resolver and will include SRV record resolution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the info. I've updated the doc.
@a11r, I believe this is ready to be merged now.