ngx-dynamic-healthcheck - Nginx upstreams healthchecks.
Module requires zone upstream directive.
- support http, tcp, ssl checks.
- support dynamic reconfiguration
- support persistance of healthcheck parameters
- optionally support LUA API for reconfiguration
- with dynamic-upstream-module automaticaly cover by healthchecks new added peers (with DNS balancing).
- TCP checks with requrest/response (for Redis for example)
- pattern matching
- disabling hosts
This module supports http and stream upstream types.
- Name
- Status
- Description
- Install
- Synopsis
- Configuration directives
- LUA API
This module is production ready.
This module provides dynamicaly configurable healthchecks.
Build nginx. All dependencies are downloaded automaticaly.
git clone git@github.com:ZigzagAK/ngx_dynamic_healthchecks.git
cd ngx_dynamic_healthchecks
./build.sh
Archive will be placed in the install folder after successful build.
http {
healthcheck fall=2 rise=2 interval=60 timeout=10000 type=tcp;
healthcheck_persistent healthcheck;
healthcheck_disable_host balancer01.domain.net:9000;
healthcheck_disable_host balancer02.domain.net:9000;
server {
listen 8888;
location = /healthcheck/get {
healthcheck_get;
}
location = /healthcheck/status {
healthcheck_status;
}
location = /healthcheck/update {
healthcheck_update;
}
}
upstream mail {
zone mail 128k;
# https://github.com/ZigzagAK/ngx_dynamic_upstream ver >= 2.X.X
dns_update 10s;
dns_ipv6 off;
server mail.ru:443 down;
check fall=2 rise=2 interval=10 timeout=10000 type=ssl;
}
upstream google {
zone google 128k;
# https://github.com/ZigzagAK/ngx_dynamic_upstream ver >= 2.X.X
dns_update 10s;
dns_ipv6 off;
server mail.ru down;
check fall=2 rise=2 interval=10 timeout=10000 type=tcp;
}
upstream test {
zone test 128k;
# https://github.com/ZigzagAK/ngx_dynamic_upstream ver >= 2.X.X
dns_update 10s;
dns_ipv6 off;
server balancer.domain.net:9000 down;
check fall=2 rise=2 interval=10 timeout=10000 type=http;
check_request_uri GET /health;
check_request_body ping;
check_response_body .*;
check_request_headers a=1 b=2;
check_response_codes 200 204;
}
}
stream {
upstream redis-ro {
zone shm_redis-ro 128k;
server 127.0.0.1:6379 down;
server 127.0.0.2:6379 down;
server 127.0.1.1:6379 down;
server 127.0.1.2:6379 down;
check fall=1 rise=2 timeout=10000 interval=10;
check_request_body "multi\r\ninfo replication\r\nget x\r\nexec\r\nquit\r\n";
check_response_body "(\+OK).*(\+QUEUED).*(\+QUEUED).*(role:slave).*(master_link_status:up).*(master_sync_in_progress:0).*(check).*(\+OK)";
}
upstream redis-rw {
zone shm_redis-rw 128k;
server 127.0.0.1:6379 down;
server 127.0.0.2:6379 down;
server 127.0.1.1:6379 down;
server 127.0.1.2:6379 down;
check fall=1 rise=2 timeout=10000 interval=10;
check_request_body "multi\r\ninfo replication\r\nset x check\r\nget x\r\nexec\r\nquit\r\n";
check_response_body "(\+OK).*(\+QUEUED).*(\+QUEUED).*(\+QUEUED).*(role:master).*(\+OK).*(check).*(\+OK)";
}
}
Global healthcheck parameters may be used in case when you want to cover all upstreams with same healthchecks.
Every parameter may be redefined on upstream level.
- syntax:
check fall=2 rise=2 timeout=1000 interval=10 keepalive=10 type=http|tcp|ssl port=<other check port> <passive> - default:
none - context:
upstream
Configure healthcheck base parameters.
If interval is 0 - no healthcheks for this upstream.
If you want change default port (got from upstream peer) you may override it with port=xxx parameter.
For example: Your service is responsible on port A and working by TCP binary protocol and you have another port opened by HTTP and returns healthcheck status.
In this case you may override peer port with separate HTTP port and setup check_request_uri and check_response_codes, check_response_body parameters.
Peer port in this case will not be check because healthcheck may be accessed on separate HTTP port.
passive parameter may be used to minimze HTTP checks. In this mode active checks are not applied when success (status < 300) responses are received from upstream peer.
- syntax:
check_request_uri GET|POST /ping - default:
none - context:
upstream
Configure http request for healthcheck.
- syntax:
check_request_headers a=1 b=2 - default:
none - context:
upstream
Configure http request headers for healthcheck.
- syntax:
check_request_body hello - default:
none - context:
upstream
Configure http request body for healthcheck.
- syntax:
check_response_codes 200 201 202 - default:
none - context:
upstream
Configure http response codes for healthcheck.
- syntax:
check_response_body .* - default:
none - context:
upstream
Configure regular expression for http response body.
- syntax:
check_persistent <folder>/off - default:
off - context:
upstream
Configure persistance for healthcheck parameters.
Parameters a stored in plain text files and may be changed online in any time without reloads. Saved parameters have higher priority than configuration parameters. They are redefine them.
To disable persistace for specific upstream you may write check_persistent off.
- syntax:
check_disable_host 1.1.1.1:3456 - default:
none - context:
upstream
Disable specific host or peer in upstream (may be changed via persistance).
- syntax:
check_exclude_host 1.1.1.1:3456 - default:
none - context:
upstream
Exclude specific host or peer in upstream from healthcheck.
- syntax:
healthcheck fall=2 rise=2 timeout=1000 interval=10 keepalive=10 type=http|tcp|ssl - default:
none - context:
http
Configure healthcheck base parameters globally.
- syntax:
healthcheck_request_uri GET|POST /ping - default:
none - context:
http
Configure http request for healthcheck globally.
- syntax:
healthcheck_request_headers a=1 b=2 - default:
none - context:
http
Configure http request headers for healthcheck globally.
- syntax:
healthcheck_request_body hello - default:
none - context:
http
Configure http request body for healthcheck globally.
- syntax:
healthcheck_response_codes 200 201 202 - default:
none - context:
http
Configure http response codes for healthcheck globally.
- syntax:
healthcheck_response_body .* - default:
none - context:
http
Configure regular expression for http response body globally.
- syntax:
healthcheck_persistent <folder>/off - default:
off - context:
http
Configure persistance for healthcheck parameters globally.
Parameters a stored in plain text files and may be changed online in any time without reloads. Saved parameters have higher priority than configuration parameters. They are redefine them.
- syntax:
healthcheck_disable_host 1.1.1.1:4567 - default:
none - context:
upstream
Disable specific host or peer in upstream (may be changed via persistance) globally.
- syntax:
healthcheck_buffer_size 32k - default:
pagesize - context:
http
Specify buffer size for sending and parsing requests and responses.
- syntax:
healthcheck_get; - context:
location
Register get healthcheck information handler.
Example output:
curl localhost:8888/healthcheck/get
{
"a":{
"rise":1,
"fall":2,
"interval":10,
"keepalive":10,
"timeout":10000,
"type":"http",
"command":{
"uri":"/health",
"method":"GET",
"headers":{"a":"1","b":"2"},
"body":"ping",
"expected":{
"body":"pong",
"codes":[200,204,201]
}
},
"disabled":0,
"off":0,
"disabled_hosts":[]
},
"google":{
"rise":2,
"fall":2,
"interval":10,
"keepalive":1,
"timeout":10000,
"type":"http",
"command":{
"uri":"/",
"method":"GET",
"headers":[],
"request_body":"",
"expected":{
"body":"",
"codes":[]
}
},
"disabled":0,
"off":0,
"disabled_hosts":[]
}
}
Arguments:
- stream=
- upstream=name
On default the handler returns information about all http upstreams. To get information about streams you may pass stream= argument to request.
To get information about specific upstream you may mass upstream=xxx agrument.
- syntax:
healthcheck_status; - context:
location
Get healthcheck online information.
Example output:
curl localhost:8888/healthcheck/status
{
"a":{
"primary":{
"127.0.0.1:9000":{
"down":1,
"fall":0,
"rise":0,
"fall_total":0,
"rise_total":0
},
"127.0.1.1:9000":{
"down":1,
"fall":0,
"rise":0,
"fall_total":0,
"rise_total":0
}
},
"backup":{
"127.0.2.1:9000":{
"down":1,
"fall":3,
"rise":0,
"fall_total":1,
"rise_total":0
}
}
},
"google":{
"primary":{
"173.194.220.102:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
},
"173.194.220.138:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
},
"173.194.220.113:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
},
"173.194.220.139:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
},
"173.194.220.100:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
},
"173.194.220.101:80":{
"down":0,
"fall":0,
"rise":2,
"fall_total":0,
"rise_total":2
}
}
}
}
Arguments:
- stream=
- upstream=name
On default the handler returns information about all http upstreams. To get information about streams you may pass stream= argument to request.
To get information about specific upstream you may mass upstream=xxx agrument.
- syntax:
healthcheck_update; - context:
location
Arguments:
- stream=
- upstream=xxx
- type=http|tcp|ssl
- fall=N
- rise=N
- timeout=ms
- interval=sec
- keepalive=N
- request_uri=URI
- request_method=GET|POST|....
- request_headers=h1:v1|h2:v2|...
- request_body=BODY
- response_codes=200|201|...
- response_body=REGEXP
- off=1|0
- disable_host=XXX.XXX.XXX.XXX:PORT
- enable_host=XXX.XXX.XXX.XXX:PORT
- disable=1|0
upstream is mandatory for all updates (ex. disable/enable hosts).
off - enable/disable healthchecks for upstream.
disable - absolutely disable app peers in upstream.
enable/disable host may be used with upstream or not. When no upstream is defined peers disabling/enabling in all upstreams.
Package ngx.healthcheck is used for manipulation http upstreams.
Package ngx.healthcheck.stream is used for manipulation stream upstreams.
Functionality of both packages is same.
syntax 1: healthcheck, error = hc.get(upstream)
syntax 2: healthcheck, error = hc.get()
context: *_by_lua*
Get healthcheck parameters.
Returns lua table on success, or error otherwise.
syntax: ok, error = hc.update(upstream, tab)
context: *_by_lua*
Update healthcheck parameters.
Example table:
{
"rise":1,
"fall":2,
"interval":10,
"keepalive":10,
"timeout":10000,
"type":"http",
"command":{
"uri":"/health",
"method":"GET",
"headers":{"a":"1","b":"2"},
"body":"ping",
"expected":{
"body":"1111",
"codes":[200,204,201]
}
},
"disabled":0,
"off":0
}
syntax: ok, error = hc.disable_host(host, disabled=1|0, [upstream])
context: *_by_lua*
Disable (permanent down) specific host in upstream (or in all upstreams).
Returns true or throws the error.
syntax: ok, error = hc.disable(upstream, disabled=1|0)
context: *_by_lua*
Disable (permanent down) all hosts in upstream.
Returns true or throws the error.
syntax: status, error = hc.status(upstream)
context: *_by_lua*
Returns runlime healthcheck information about peers in upstream.