Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add nacos support #3820

Merged
merged 66 commits into from
Apr 21, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
baab627
add nacos support
benx203 Mar 13, 2021
e053355
add docs
benx203 Mar 15, 2021
5b14149
fix lint warnings
benx203 Mar 18, 2021
613c221
fix test
benx203 Mar 18, 2021
f0b0942
fix nacos
benx203 Mar 18, 2021
221ee3a
fix nacos test script
benx203 Mar 19, 2021
8e3e6a5
fix
benx203 Mar 19, 2021
69055a0
fix nacos - support auth
benx203 Mar 19, 2021
e412a70
fix
benx203 Mar 19, 2021
9b48317
fix
benx203 Mar 19, 2021
d8f819b
fix
benx203 Mar 19, 2021
46fe9fd
fix
benx203 Mar 20, 2021
9e51570
reindex
benx203 Mar 20, 2021
4a0dac6
fix env script
benx203 Mar 21, 2021
bbc3f1f
fix env script
benx203 Mar 21, 2021
b153240
fix env script
benx203 Mar 21, 2021
962be01
fix test script
benx203 Mar 22, 2021
26de944
fix test script
benx203 Mar 22, 2021
c1e6f00
fix
benx203 Mar 22, 2021
95ab302
fix
benx203 Mar 22, 2021
63eba52
fix
benx203 Mar 23, 2021
df1d54f
fix
benx203 Mar 23, 2021
3ea6a66
fix
benx203 Mar 29, 2021
d1f3bef
fix
benx203 Mar 29, 2021
ec0c16d
Merge branch 'master' of https://github.com/apache/apisix
benx203 Apr 1, 2021
397a197
fix
benx203 Apr 1, 2021
dcb76c2
fix
benx203 Apr 1, 2021
14de476
fix
benx203 Apr 1, 2021
b53f4e9
fix
benx203 Apr 2, 2021
dccd644
fix
benx203 Apr 2, 2021
33e7c2e
fix
benx203 Apr 2, 2021
ddff0f5
fix
benx203 Apr 2, 2021
3ebf626
fix
benx203 Apr 2, 2021
147c088
modify test
benx203 Apr 2, 2021
d55effc
fix
benx203 Apr 2, 2021
ea93f5e
fix
benx203 Apr 6, 2021
922ac50
trigger run test script
benx203 Apr 6, 2021
5a91e81
fix test
benx203 Apr 7, 2021
0d1ef67
fix
benx203 Apr 7, 2021
7398fbb
fix
benx203 Apr 7, 2021
74f5512
fix
benx203 Apr 7, 2021
b7d356c
fix
benx203 Apr 7, 2021
4fa00d7
rerun test
benx203 Apr 7, 2021
638cdc9
fix
benx203 Apr 7, 2021
e24d641
fix
benx203 Apr 7, 2021
dce8163
fix test
benx203 Apr 8, 2021
04a42a2
fix
benx203 Apr 8, 2021
846fc86
fix test
benx203 Apr 8, 2021
952aa04
fix test
benx203 Apr 8, 2021
5ea5414
fix
benx203 Apr 8, 2021
e1a9f13
Merge remote-tracking branch 'upstream/master'
benx203 Apr 8, 2021
6bbcaf1
debug test
benx203 Apr 8, 2021
f150a72
debug test
benx203 Apr 8, 2021
9ac15df
debug test
benx203 Apr 8, 2021
c97c0a2
fork the repo to prevent the original repo deleted by accident
spacewander Apr 9, 2021
4936b19
tweak
spacewander Apr 9, 2021
fb69097
WIP
spacewander Apr 9, 2021
6171c2e
Merge branch 'master' into benx203_master
spacewander Apr 14, 2021
83d7500
fix lint
spacewander Apr 14, 2021
5af1d9e
fix
benx203 Apr 19, 2021
69408be
fix
benx203 Apr 19, 2021
dfd6921
fix
benx203 Apr 19, 2021
8b196f3
fix
benx203 Apr 19, 2021
09767bd
fix
benx203 Apr 19, 2021
660d2b6
trigger test
benx203 Apr 20, 2021
407fb42
style
spacewander Apr 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ A/B testing, canary release, blue-green deployment, limit rate, defense against
- **OPS friendly**

- OpenTracing: support [Apache Skywalking](docs/en/latest/plugins/skywalking.md) and [Zipkin](docs/en/latest/plugins/zipkin.md)
- works with external service discovery:In addition to the built-in etcd, it also supports `Consul` and `Nacos` [DNS discovery mode](https://github.com/apache/apisix/issues/1731#issuecomment-646392129), and [Eureka](docs/en/latest/discovery.md)
- works with external service discovery:In addition to the built-in etcd, it also supports `Consul` and [Nacos](docs/en/latest/discovery.md), and [Eureka](docs/en/latest/discovery.md)
- Monitoring And Metrics: [Prometheus](docs/en/latest/plugins/prometheus.md)
- Clustering: APISIX nodes are stateless, creates clustering of the configuration center, please refer to [etcd Clustering Guide](https://etcd.io/docs/v3.4.0/op-guide/clustering/).
- High availability: Support to configure multiple etcd addresses in the same cluster.
Expand Down
132 changes: 61 additions & 71 deletions apisix/discovery/nacos.lua
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,16 @@
--

local require = require
local local_conf = require("apisix.core.config_local").local_conf()
local http = require("resty.http")
local core = require("apisix.core")
local local_conf = require('apisix.core.config_local').local_conf()
local http = require('resty.http')
local core = require('apisix.core')
local ipairs = ipairs
local type = type
local math = math
local math_random = math.random
local error = error
local ngx = ngx
local ngx_re = require("ngx.re")
local ngx_re = require('ngx.re')
local ngx_timer_at = ngx.timer.at
local ngx_timer_every = ngx.timer.every
local string = string
Expand All @@ -35,29 +35,34 @@ local str_find = core.string.find
local log = core.log

local default_weight
local applications
local auth_path = "auth/login"
local instance_list_path = "ns/instance/list?healthyOnly=true&serviceName="
local applications = {}
local auth_path = 'auth/login'
local instance_list_path = 'ns/instance/list?healthyOnly=true&serviceName='

local host_pattern = [[^http(s)?:\/\/[a-zA-Z0-9-_.:\@]+$]]
local prefix_pattern = [[^[\/a-zA-Z0-9-_.]+$]]
local schema = {
type = "object",
type = 'object',
properties = {
host = {
type = "array",
type = 'array',
minItems = 1,
items = {
type = "string",
type = 'string',
pattern = host_pattern,
minLength = 2,
maxLength = 100,
},
},
fetch_interval = {type = "integer", minimum = 1, default = 30},
prefix = {type = "string", default = "/nacos/v1/"},
weight = {type = "integer", minimum = 1, default = 100},
fetch_interval = {type = 'integer', minimum = 1, default = 30},
prefix = {type = 'string', pattern = prefix_pattern, maxLength = 100, default = '/nacos/v1/'},
weight = {type = 'integer', minimum = 1, default = 100},
timeout = {
type = "object",
type = 'object',
properties = {
connect = {type = "integer", minimum = 1, default = 2000},
send = {type = "integer", minimum = 1, default = 2000},
read = {type = "integer", minimum = 1, default = 5000},
connect = {type = 'integer', minimum = 1, default = 2000},
send = {type = 'integer', minimum = 1, default = 2000},
read = {type = 'integer', minimum = 1, default = 5000},
},
default = {
connect = 2000,
Expand All @@ -66,19 +71,17 @@ local schema = {
}
},
},
required = {"host"}
required = {'host'}
}


local _M = {
version = 0.1,
}
local _M = {}


local function request(request_uri, path, body, method, basic_auth)
local url = request_uri .. path
log.info("request url:", url)
local headers = core.table.new(0, 0)
log.info('request url:', url)
local headers = {}
headers['Accept'] = 'application/json'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the string object, I think we use ", it is easier for reading.

headers["Accept"] = "application/json"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and please fix the similar points


if basic_auth then
Expand All @@ -99,8 +102,8 @@ local function request(request_uri, path, body, method, basic_auth)
local connect_timeout = timeout.connect
local send_timeout = timeout.send
local read_timeout = timeout.read
log.info("connect_timeout:", connect_timeout, ", send_timeout:", send_timeout,
", read_timeout:", read_timeout)
log.info('connect_timeout:', connect_timeout, ', send_timeout:', send_timeout,
', read_timeout:', read_timeout)
httpc:set_timeouts(connect_timeout, send_timeout, read_timeout)
local res, err = httpc:request_uri(url, {
method = method,
Expand All @@ -113,7 +116,7 @@ local function request(request_uri, path, body, method, basic_auth)
end

if not res.body or res.status ~= 200 then
return nil, "status = " .. res.status
return nil, 'status = ' .. res.status
end

local json_str = res.body
Expand All @@ -126,40 +129,41 @@ end


local function get_url(request_uri, path)
return request(request_uri, path, nil, "GET", nil)
return request(request_uri, path, nil, 'GET', nil)
end


local function post_url(request_uri, path, body)
return request(request_uri, path, body, "POST", nil)
return request(request_uri, path, body, 'POST', nil)
end


local function get_token_param(base_uri, username, password)
if username and password then
local data, err = post_url(base_uri, auth_path .. "?username=" .. username
.. "&password=" .. password, nil)
if err then
log.error("nacos login fail:", username, " ", password, " desc:", err)
return nil, err
end
return "&accessToken=" .. data.accessToken
if not username or not password then
return ''
end
return ""

local args = { username = username, password = password}
local data, err = post_url(base_uri, auth_path .. '?' .. ngx.encode_args(args), nil)
if err then
log.error('nacos login fail:', username, ' ', password, ' desc:', err)
return nil, err
end
return '&accessToken=' .. data.accessToken
end


local function get_base_uri()
local host = local_conf.discovery.nacos.host
-- TODO Add health check to get healthy nodes.
local url = host[math_random(#host)]
local auth_idx = str_find(url, "@")
local auth_idx = str_find(url, '@', 1, true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core.string.find is already the plain string find... 😅
Not all the suggestion from the reviewer is correct.

local username, password
if auth_idx then
local protocol_idx = str_find(url, "://")
local protocol_idx = str_find(url, '://', 1, true)
local protocol = string_sub(url, 1, protocol_idx + 2)
local user_and_password = string_sub(url, protocol_idx + 3, auth_idx - 1)
local arr = ngx_re.split(user_and_password, ":")
local arr = ngx_re.split(user_and_password, ':')
if #arr == 2 then
username = arr[1]
password = arr[2]
Expand All @@ -172,16 +176,16 @@ local function get_base_uri()
url = url .. local_conf.discovery.nacos.prefix
end

if str_byte(url, #url) ~= str_byte("/") then
url = url .. "/"
if str_byte(url, #url) ~= str_byte('/') then
url = url .. '/'
end

return url, username, password
end


local function iter_and_add_service(services, values)
if not values then
if not values then
return
end

Expand All @@ -198,10 +202,10 @@ local function iter_and_add_service(services, values)
up = conf
end

if up.discovery_type == "nacos" then
if up.discovery_type == 'nacos' then
core.table.insert(services, up.service_name)
end
::CONTINUE::
::CONTINUE::
end
end

Expand All @@ -210,9 +214,9 @@ local function get_nacos_services()
local services = {}

-- here we use lazy load to work around circle dependency
local get_upstreams = require("apisix.upstream").upstreams
local get_routes = require("apisix.router").http_routes
local get_services = require("apisix.http.service").services
local get_upstreams = require('apisix.upstream').upstreams
local get_routes = require('apisix.router').http_routes
local get_services = require('apisix.http.service').services

local values = get_upstreams()
iter_and_add_service(services, values)
Expand All @@ -229,14 +233,11 @@ local function fetch_full_registry(premature)
return
end

local up_apps = core.table.new(0, 0)
local up_apps = {}
local base_uri, username, password = get_base_uri()
local token_param, err = get_token_param(base_uri, username, password)
if err then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to writhe this error message to error log

log.error("get_token_param error:", err)
if not applications then
applications = up_apps
end
log.error('get_token_param error:', err)
return
end

Expand All @@ -250,17 +251,14 @@ local function fetch_full_registry(premature)
for _, service_name in ipairs(infos) do
data, err = get_url(base_uri, instance_list_path .. service_name .. token_param)
spacewander marked this conversation as resolved.
Show resolved Hide resolved
if err then
log.error("get_url:", instance_list_path, " err:", err)
if not applications then
applications = up_apps
end
log.error('get_url:', instance_list_path, ' err:', err)
return
end

for _, host in ipairs(data.hosts) do
local nodes = up_apps[service_name]
if not nodes then
nodes = core.table.new(0, 0)
nodes = {}
up_apps[service_name] = nodes
end
core.table.insert(nodes, {
Expand All @@ -275,41 +273,33 @@ end


function _M.nodes(service_name)
local logged = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original code is good. We just need to set a 5s limitation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just keep simple.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lazy init is important as the sync is slow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK,i rollback the code.

while not applications do
if not logged then
log.warn('wait init')
logged = true
end
ngx.sleep(0.1)
end
return applications[service_name]
end


function _M.init_worker()
if not local_conf.discovery.nacos or
not local_conf.discovery.nacos.host or #local_conf.discovery.nacos.host == 0 then
error("do not set nacos.host")
error('do not set nacos.host')
return
end

local ok, err = core.schema.check(schema, local_conf.discovery.nacos)
if not ok then
error("invalid nacos configuration: " .. err)
error('invalid nacos configuration: ' .. err)
return
end
default_weight = local_conf.discovery.nacos.weight
log.info("default_weight:", default_weight)
log.info('default_weight:', default_weight)
local fetch_interval = local_conf.discovery.nacos.fetch_interval
log.info("fetch_interval:", fetch_interval)
log.info('fetch_interval:', fetch_interval)
ngx_timer_at(0, fetch_full_registry)
ngx_timer_every(fetch_interval, fetch_full_registry)
end


function _M.dump_data()
return {config = local_conf.discovery.nacos, services = applications or {}}
return {config = local_conf.discovery.nacos, services = applications}
end


Expand Down
2 changes: 2 additions & 0 deletions docs/zh/latest/discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ title: 集成服务发现注册中心

Consul 的支持见 [基于 Consul 的服务支持发现](../../en/latest/discovery/consul_kv.md)

Nacos 的支持见 [基于 Nacos 的服务支持发现](../../en/latest/discovery/nacos.md)

Eureka 的支持方式见下文。

## 如何扩展注册中心?
Expand Down