Skip to content

Commit

Permalink
Fix issues with Kubernetes (recursion) and port numbers for SRV (#3)
Browse files Browse the repository at this point in the history 
* fix issues with Kubernetes dns recursive SRV record
* fix Kubernetes issue with SRV ports being '0'
* updated for release 0.3.1
  • Loading branch information
@Tieske
Tieske authored Feb 22, 2017
1 parent ac872e4 commit 705f3e9
Show file tree
Hide file tree
Showing 9 changed files with 207 additions and 156 deletions.
12 changes: 11 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Features
Copyright and license
=====================

Copyright: 2016 Mashape, Inc.
Copyright: (c) 2016-2017 Mashape, Inc.

Author: Thijs Schreijer

Expand All @@ -35,6 +35,16 @@ use the `rbusted` script.
History
=======

###0.3.1 (22-Feb-2017) Bugfix

- Kubernetes dns returns an SRV record for individual nodes, where the target
is the same name again (hence causing a recursive loop). Now those entries
will be removed, and if nothing is left, it will fail the SRV lookup, causing
a fall-through to the next record type.
- Kubernetes tends to return a port of 0 if none is provided/set, hence the
`toip()` function now ignores a `port=0` and falls back on the port passed
in.

###0.3.0 (8-Nov-2016) Major breaking update

- breaking: renamed a lot of things; method names, module names, etc. pretty
Expand Down
4 changes: 2 additions & 2 deletions docs/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ <h2>Topics</h2>
</div> <!-- id="content" -->
</div> <!-- id="main" -->
<div id="about">
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.5</a></i>
<i style="float:right;">Last updated 2016-11-08 22:07:36 </i>
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.6</a></i>
<i style="float:right;">Last updated 2017-02-22 13:40:23 </i>
</div> <!-- id="about" -->
</div> <!-- id="container" -->
</body>
Expand Down
58 changes: 29 additions & 29 deletions docs/modules/resty.dns.balancer.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ <h2 class="section-header "><a name="Functions"></a>Functions</h2>
<h3>Parameters:</h3>
<ul>
<li><span class="parameter">hostname</span>
hostname to add (note: not validated, as long as it&rsquo;s a
hostname to add (note: not validated, as long as it's a
string it will be accepted, but remain unresolved!)
</li>
<li><span class="parameter">port</span>
Expand Down Expand Up @@ -214,7 +214,8 @@ <h3>Returns:</h3>
<strong>new (opts)</strong>
</dt>
<dd>
<p>Creates a new balancer. The balancer is based on a wheel with slots. The

<p>Creates a new balancer. The balancer is based on a wheel with slots. The
slots will be randomly distributed over the targets. The number of slots
assigned will be relative to the weight.</p>

Expand All @@ -225,35 +226,34 @@ <h3>Returns:</h3>
<p> The options table has the following fields;</p>

<ul>
<li><code>hosts</code> (optional) containing hostnames, ports and weights. If omitted,
ports and weights default respectively to 80 and 10. The list will be sorted
before being added, so the order of entry is deterministic.</li>
<li><code>wheelSize</code> (optional) for total number of slots in the balancer, if omitted
the size of <code>order</code> is used, or 1000 if <code>order</code> is not provided. It is important
to have enough slots to keep the ring properly randomly distributed. If there
are to few slots for the number of targets then the load distribution might
become to coarse. Consider the maximum number of targets expected, as new
hosts can be dynamically added, and dns renewals might yield larger record
sets. The <code>wheelSize</code> cannot be altered, only a new wheel can be created, but
then all consistency would be lost. On a similar note; making it too big,
will have a performance impact when the wheel is modified as too many slots
will have to be moved between targets. A value of 50 to 200 slots per entry
seems about right.</li>
<li><code>order</code> (optional) if given, a list of random numbers, size <code>wheelSize</code>, used to
randomize the wheel. This entry is solely to support multiple servers with
the same consistency, as it allows to use the same randomization on each
server, and hence the same slot assignment. Duplicates are not allowed in
the list.</li>
<li><code>dns</code> (required) a configured <a href="../modules/resty.dns.client.html">dns.client</a> object for querying the dns server.</li>
<li><code>requery</code> (optional) interval of requerying the dns server for previously
failed queries. Defaults to 1 if omitted (in seconds)</li>
<li><code>ttl0</code> (optional) Maximum lifetime for records inserted with ttl=0, to verify
the ttl is still 0. Defaults to 60 if omitted (in seconds)</li>
<li><code>hosts</code> (optional) containing hostnames, ports and weights. If omitted,
ports and weights default respectively to 80 and 10. The list will be sorted
before being added, so the order of entry is deterministic.</li>
<li><code>wheelSize</code> (optional) for total number of slots in the balancer, if omitted
the size of <code>order</code> is used, or 1000 if <code>order</code> is not provided. It is important
to have enough slots to keep the ring properly randomly distributed. If there
are to few slots for the number of targets then the load distribution might
become to coarse. Consider the maximum number of targets expected, as new
hosts can be dynamically added, and dns renewals might yield larger record
sets. The <code>wheelSize</code> cannot be altered, only a new wheel can be created, but
then all consistency would be lost. On a similar note; making it too big,
will have a performance impact when the wheel is modified as too many slots
will have to be moved between targets. A value of 50 to 200 slots per entry
seems about right.</li>
<li><code>order</code> (optional) if given, a list of random numbers, size <code>wheelSize</code>, used to
randomize the wheel. This entry is solely to support multiple servers with
the same consistency, as it allows to use the same randomization on each
server, and hence the same slot assignment. Duplicates are not allowed in
the list.</li>
<li><code>dns</code> (required) a configured <a href="../modules/resty.dns.client.html">dns.client</a> object for querying the dns server.</li>
<li><code>requery</code> (optional) interval of requerying the dns server for previously
failed queries. Defaults to 1 if omitted (in seconds)</li>
<li><code>ttl0</code> (optional) Maximum lifetime for records inserted with ttl=0, to verify
the ttl is still 0. Defaults to 60 if omitted (in seconds)</li>
</ul>




<h3>Parameters:</h3>
<ul>
<li><span class="parameter">opts</span>
Expand Down Expand Up @@ -319,8 +319,8 @@ <h3>Returns:</h3>
</div> <!-- id="content" -->
</div> <!-- id="main" -->
<div id="about">
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.5</a></i>
<i style="float:right;">Last updated 2016-11-08 22:07:36 </i>
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.6</a></i>
<i style="float:right;">Last updated 2017-02-22 13:40:23 </i>
</div> <!-- id="about" -->
</div> <!-- id="container" -->
</body>
Expand Down
103 changes: 52 additions & 51 deletions docs/modules/resty.dns.client.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ <h1>lua-resty-dns-client</h1>

<h2>Contents</h2>
<ul>
<li><a href="#Caching">Caching</a></li>
<li><a href="#Resolving">Resolving</a></li>
<li><a href="#Socket_functions">Socket functions</a></li>
<li><a href="#Caching">Caching </a></li>
<li><a href="#Resolving">Resolving </a></li>
<li><a href="#Socket_functions">Socket functions </a></li>
</ul>


Expand All @@ -55,22 +55,24 @@ <h2>Topics</h2>

<h1>Module <code>resty.dns.client</code></h1>
<p>DNS client.</p>
<p><p> Works with OpenResty only. Requires the <a href="https://github.com/openresty/lua-resty-dns"><code>lua-resty-dns</code></a> module.</p>
<p>


<p> Works with OpenResty only. Requires the <a href="https://github.com/openresty/lua-resty-dns"><code>lua-resty-dns</code></a> module.</p>

<p> <em>NOTES</em>: </p>

<ol>
<li>parsing the config files upon initialization uses blocking i/o, so use with
care. See <a href="../modules/resty.dns.client.html#init">init</a> for details.</li>
<li>All returned records are directly from the cache. <em>So do not modify them!</em>
If you need to, copy them first.</li>
<li>TTL for records is the TTL returned by the server at the time of fetching
and won&rsquo;t be updated while the client serves the records from its cache.</li>
<li>resolving IPv4 (A-type) and IPv6 (AAAA-type) addresses is explicitly supported. If
the hostname to be resolved is a valid IP address, it will be cached with a ttl of
10 years. So the user doesn&rsquo;t have to check for ip adresses.</li>
<li>parsing the config files upon initialization uses blocking i/o, so use with
care. See <a href="../modules/resty.dns.client.html#init">init</a> for details.</li>
<li>All returned records are directly from the cache. <em>So do not modify them!</em>
If you need to, copy them first.</li>
<li>TTL for records is the TTL returned by the server at the time of fetching
and won't be updated while the client serves the records from its cache.</li>
<li>resolving IPv4 (A-type) and IPv6 (AAAA-type) addresses is explicitly supported. If
the hostname to be resolved is a valid IP address, it will be cached with a ttl of
10 years. So the user doesn't have to check for ip adresses.</li>
</ol>

</p>
<h3>Info:</h3>
<ul>
Expand All @@ -80,14 +82,14 @@ <h3>Info:</h3>
</ul>


<h2><a href="#Caching">Caching</a></h2>
<h2><a href="#Caching">Caching </a></h2>
<table class="function_list">
<tr>
<td class="name" nowrap><a href="#purgeCache">purgeCache (touched)</a></td>
<td class="summary">Cleanup the DNS client cache.</td>
</tr>
</table>
<h2><a href="#Resolving">Resolving</a></h2>
<h2><a href="#Resolving">Resolving </a></h2>
<table class="function_list">
<tr>
<td class="name" nowrap><a href="#init">init (options)</a></td>
Expand All @@ -106,7 +108,7 @@ <h2><a href="#Resolving">Resolving</a></h2>
<td class="summary">Resolves to an IP and port number.</td>
</tr>
</table>
<h2><a href="#Socket_functions">Socket functions</a></h2>
<h2><a href="#Socket_functions">Socket functions </a></h2>
<table class="function_list">
<tr>
<td class="name" nowrap><a href="#connect">connect (sock, host, port, opts)</a></td>
Expand All @@ -122,7 +124,7 @@ <h2><a href="#Socket_functions">Socket functions</a></h2>
<br/>


<h2 class="section-header has-description"><a name="Caching"></a>Caching</h2>
<h2 class="section-header has-description"><a name="Caching"></a>Caching </h2>

<div class="section-description">
The cache will not update the <code>ttl</code> field. So every time the same record
Expand Down Expand Up @@ -163,7 +165,7 @@ <h3>Returns:</h3>

</dd>
</dl>
<h2 class="section-header has-description"><a name="Resolving"></a>Resolving</h2>
<h2 class="section-header has-description"><a name="Resolving"></a>Resolving </h2>

<div class="section-description">
When resolving names, queries will be synchronized, such that only a single
Expand Down Expand Up @@ -248,7 +250,8 @@ <h3>Usage:</h3>
<strong>resolve (qname, r_opts, dnsCacheOnly, r)</strong>
</dt>
<dd>
<p>Resolve a name.<br/>

<p>Resolve a name. <br/>
If <code>r_opts.qtype</code> is given, then it will fetch that specific type only. If
<code>r_opts.qtype</code> is not provided, then it will try to resolve
the name using the record types, in the order as provided to <a href="../modules/resty.dns.client.html#init">init</a>.</p>
Expand All @@ -260,20 +263,19 @@ <h3>Usage:</h3>
it (the first time) as;</p>

<ul>
<li>SRV, </li>
<li>then A, </li>
<li>then AAAA (success),</li>
<li>then CNAME (after AAAA success, this will not be tried)</li>
<li>SRV, </li>
<li>then A, </li>
<li>then AAAA (success),</li>
<li>then CNAME (after AAAA success, this will not be tried)</li>
</ul>


<p> A second lookup will now try (assuming the cached entry expired);</p>

<ul>
<li>AAAA (as it was the last successful lookup),</li>
<li>then SRV, </li>
<li>then A,</li>
<li>then CNAME.</li>
<li>AAAA (as it was the last successful lookup),</li>
<li>then SRV, </li>
<li>then A,</li>
<li>then CNAME.</li>
</ul>


Expand All @@ -289,7 +291,7 @@ <h3>Parameters:</h3>
<a href="https://github.com/openresty/lua-resty-dns">OpenResty docs</a> for more options.
</li>
<li><span class="parameter">dnsCacheOnly</span>
Only check the cache, won&rsquo;t do server lookups
Only check the cache, won't do server lookups
(will not invalidate any ttl expired data and will hence possibly return
expired data)
</li>
Expand Down Expand Up @@ -323,7 +325,7 @@ <h3>Returns:</h3>
A server error table is returned as <code>nil+error+r</code> (where <a href="https://www.lua.org/manual/5.1/manual.html#pdf-error">error</a> is a string
extracted from the server error table).
An empty response is returned as <code>response+error+r</code> (where <a href="https://www.lua.org/manual/5.1/manual.html#pdf-error">error</a> is
&lsquo;dns query returned no results&rsquo;).
'dns query returned no results').



Expand All @@ -349,7 +351,8 @@ <h3>Usage:</h3>
<strong>toip (qname, port, dnsCacheOnly, r)</strong>
</dt>
<dd>
<p>Resolves to an IP and port number.

<p>Resolves to an IP and port number.
Builds on top of <a href="../modules/resty.dns.client.html#resolve">resolve</a>, but will also further dereference SRV type records.</p>

<p> When calling multiple times on cached records, it will apply load-balancing
Expand All @@ -360,31 +363,29 @@ <h3>Usage:</h3>

<p> <strong>Example</strong>;</p>

<p> SRV record for &ldquo;my.domain.com&rdquo;, containing 2 entries (this is the 1st level);</p>
<p> SRV record for "my.domain.com", containing 2 entries (this is the 1st level);</p>

<ul>
<li> <code>target = 127.0.0.1, port = 80, weight = 10</code></li>
<li> <code>target = &quot;other.domain.com&quot;, port = 8080, weight = 5</code></li>
<li><code>target = 127.0.0.1, port = 80, weight = 10</code></li>
<li><code>target = &quot;other.domain.com&quot;, port = 8080, weight = 5</code></li>
</ul>


<p> A record for &ldquo;other.domain.com&rdquo;, containing 2 entries (this is the 2nd level);</p>
<p> A record for "other.domain.com", containing 2 entries (this is the 2nd level);</p>

<ul>
<li> <code>ip = 127.0.0.2</code></li>
<li> <code>ip = 127.0.0.3</code></li>
<li><code>ip = 127.0.0.2</code></li>
<li><code>ip = 127.0.0.3</code></li>
</ul>


<p> Now calling <code>local ip, port = toip(&quot;my.domain.com&quot;, 123)</code> in a row 6 times will result in;</p>

<ul>
<li> <code>127.0.0.1, 80</code></li>
<li> <code>127.0.0.2, 8080</code> (port from SRV, 1st IP from A record)</li>
<li> <code>127.0.0.1, 80</code> (completes WRR 1st level, 1st run)</li>
<li> <code>127.0.0.3, 8080</code> (port from SRV, 2nd IP from A record, completes RR 2nd level)</li>
<li> <code>127.0.0.1, 80</code></li>
<li> <code>127.0.0.1, 80</code> (completes WRR 1st level, 2nd run, with different order as WRR is randomized)</li>
<li><code>127.0.0.1, 80</code></li>
<li><code>127.0.0.2, 8080</code> (port from SRV, 1st IP from A record)</li>
<li><code>127.0.0.1, 80</code> (completes WRR 1st level, 1st run)</li>
<li><code>127.0.0.3, 8080</code> (port from SRV, 2nd IP from A record, completes RR 2nd level)</li>
<li><code>127.0.0.1, 80</code></li>
<li><code>127.0.0.1, 80</code> (completes WRR 1st level, 2nd run, with different order as WRR is randomized)</li>
</ul>


Expand All @@ -397,10 +398,10 @@ <h3>Parameters:</h3>
</li>
<li><span class="parameter">port</span>
(optional) default port number to return if none was found in
the lookup chain (only SRV records carry port information)
the lookup chain (only SRV records carry port information, SRV with <code>port=0</code> will be ignored)
</li>
<li><span class="parameter">dnsCacheOnly</span>
Only check the cache, won&rsquo;t do server lookups (will
Only check the cache, won't do server lookups (will
not invalidate any ttl expired data and will hence possibly return expired data)
</li>
<li><span class="parameter">r</span>
Expand All @@ -421,7 +422,7 @@ <h3>Returns:</h3>

</dd>
</dl>
<h2 class="section-header "><a name="Socket_functions"></a>Socket functions</h2>
<h2 class="section-header "><a name="Socket_functions"></a>Socket functions </h2>

<dl class="function">
<dt>
Expand Down Expand Up @@ -502,8 +503,8 @@ <h3>Returns:</h3>
</div> <!-- id="content" -->
</div> <!-- id="main" -->
<div id="about">
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.5</a></i>
<i style="float:right;">Last updated 2016-11-08 22:07:36 </i>
<i>generated by <a href="http://github.com/stevedonovan/LDoc">LDoc 1.4.6</a></i>
<i style="float:right;">Last updated 2017-02-22 13:40:23 </i>
</div> <!-- id="about" -->
</div> <!-- id="container" -->
</body>
Expand Down
Loading

0 comments on commit 705f3e9

Please sign in to comment.