Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add control api for discovery module #3742

Merged
merged 6 commits into from
Mar 8, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions apisix/control/router.lua
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,10 @@ local router = require("apisix.utils.router")
local builtin_v1_routes = require("apisix.control.v1")
local plugin_mod = require("apisix.plugin")
local core = require("apisix.core")

local str_sub = string.sub
local ipairs = ipairs
local pairs = pairs
local type = type
local ngx = ngx
local get_method = ngx.req.get_method
Expand Down Expand Up @@ -66,6 +68,38 @@ function fetch_control_api_router()
end
end

local discovery_type = require("apisix.core.config_local").local_conf().discovery
if discovery_type then
local discovery = require("apisix.discovery.init").discovery
local dump_apis = {}
for key, _ in pairs(discovery_type) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we can use for key, dis_mod in pairs(discovery) do directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if use for key, dis_mod in pairs(discovery) do, the dis_mod is function type, not a table object.

So, we can't do it.

local dis_mod = discovery[key]
-- if discovery module has control_api method, support it
local api_fun = dis_mod.control_api
if api_fun then
local api_route = api_fun()
register_api_routes(routes, api_route)
end

local dump_data = dis_mod.dump_data
if dump_data then
local item = {
methods = {"GET"},
uris = {"/v1/discovery/" .. key .. "/dump"},
handler = function()
return 200, dump_data()
end
}
core.table.insert(dump_apis, item)
end
end

if #dump_apis > 0 then
core.log.notice("dump_apis: ", core.json.encode(dump_apis, true))
register_api_routes(routes, dump_apis)
end
end

core.table.clear(v1_routes)
register_api_routes(v1_routes, builtin_v1_routes)

Expand Down
1 change: 1 addition & 0 deletions apisix/discovery/consul_kv.lua
Original file line number Diff line number Diff line change
Expand Up @@ -410,4 +410,5 @@ function _M.dump_data()
return {config = local_conf.discovery.consul_kv, services = applications}
end


return _M
5 changes: 5 additions & 0 deletions apisix/discovery/eureka.lua
Original file line number Diff line number Diff line change
Expand Up @@ -252,4 +252,9 @@ function _M.init_worker()
end


function _M.dump_data()
return {config = local_conf.discovery.eureka, services = applications}
end


return _M
37 changes: 36 additions & 1 deletion docs/en/latest/discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ title: Integration service discovery registry
* [**Initial service discovery**](#initial-service-discovery)
* [**Configuration for Eureka**](#configuration-for-eureka)
* [**Upstream setting**](#upstream-setting)
* [**Embedded control api for debugging**](#embedded-control-api-for-debugging)
* [**Discovery modules**](#discovery-modules)

## Summary
Expand Down Expand Up @@ -66,7 +67,9 @@ It is very easy for APISIX to extend the discovery client, the basic steps are a

2. Implement the `_M. init_worker()` function for initialization and the `_M. nodes(service_name)` function for obtaining the list of service instance nodes;

3. Convert the registry data into data in APISIX;
3. If you need the discovery module to export the debugging information online, implement the `_M. dump_data()` function;
yongboy marked this conversation as resolved.
Show resolved Hide resolved

4. Convert the registry data into data in APISIX;

### the example of Eureka

Expand All @@ -92,6 +95,11 @@ Then implement the `_M.init_worker()` function for initialization and the `_M.no
end


function _M.dump_data()
... ...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an example return data shoul be written here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example ?

  function _M.dump_data()
      return {config = your_config, services = your_services, other = ... } 
  end

end


return _M
```

Expand Down Expand Up @@ -256,3 +264,30 @@ $ curl http://127.0.0.1:9080/apisix/admin/routes/2 -H 'X-API-KEY: edd1c9f034335f
Suppose both A-SERVICE and B-SERVICE provide a `/test` API. The above configuration allows access to A-SERVICE's `/test` API through `/a/test` and B-SERVICE's `/test` API through `/b/test`.

**Notice**:When configuring `upstream.service_name`, `upstream.nodes` will no longer take effect, but will be replaced by 'nodes' obtained from the registry.

## Embedded control api for debugging

Sometimes we need the discovery client to export online data snapshot in memory when running for debugging, and if you implement the `_M. dump_data()` function:

```lua
function _M.dump_data()
return {config = local_conf.discovery.eureka, services = applications}
end
```

Then you can call its control api as below:

```shell
GET /v1/discovery/{discovery_type}/dump
```

eg:

```shell
curl http://127.0.0.1:9090/v1/discovery/eureka/dump
```

## Discovery modules

- eureka
- [Consul KV](discovery/consul_kv.md)
75 changes: 74 additions & 1 deletion docs/en/latest/discovery/consul_kv.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,4 +147,77 @@ You could find more usage in the `apisix/t/discovery/consul_kv.t` file.

## Debugging API

TODO
It also offers control api for debugging:

```shell
GET /v1/discovery/consul_kv/dump
```

For example:

```shell
# curl http://127.0.0.1:9090/v1/discovery/consul_kv/dump | jq
{
"config": {
"fetch_interval": 3,
"timeout": {
"wait": 60,
"connect": 6000,
"read": 6000
},
"prefix": "upstreams",
"weight": 1,
"servers": [
"http://172.19.5.30:8500",
"http://172.19.5.31:8500"
],
"keepalive": true,
"default_service": {
"host": "172.19.5.11",
"port": 8899,
"metadata": {
"fail_timeout": 1,
"weigth": 1,
"weight": 1,
"max_fails": 1
}
},
"skip_keys": [
"upstreams/myapi/gateway/apisix/"
]
},
"services": {
"http://172.19.5.31:8500/v1/kv/upstreams/webpages/": [
{
"host": "127.0.0.1",
"port": 30513,
"weight": 1
},
{
"host": "127.0.0.1",
"port": 30514,
"weight": 1
}
],
"http://172.19.5.30:8500/v1/kv/upstreams/1614480/grpc/": [
{
"host": "172.19.5.51",
"port": 50051,
"weight": 1
}
],
"http://172.19.5.30:8500/v1/kv/upstreams/webpages/": [
{
"host": "127.0.0.1",
"port": 30511,
"weight": 1
},
{
"host": "127.0.0.1",
"port": 30512,
"weight": 1
}
]
}
}
```
102 changes: 102 additions & 0 deletions t/control/discovery.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
use t::APISIX 'no_plan';

repeat_each(1);
no_long_string();
no_root_location();
no_shuffle();
log_level("info");


our $yaml_config = <<_EOC_;
apisix:
enable_control: true
node_listen: 1984
config_center: yaml
enable_admin: false

discovery:
eureka:
host:
- "http://127.0.0.1:8761"
prefix: "/eureka/"
fetch_interval: 10
weight: 80
timeout:
connect: 1500
send: 1500
read: 1500
consul_kv:
servers:
- "http://127.0.0.1:8500"
- "http://127.0.0.1:8600"
dns:
servers:
- "127.0.0.1:1053"
_EOC_


run_tests();

__DATA__

=== TEST 1: test consul_kv dump_data api
--- yaml_config eval: $::yaml_config
--- request
GET /v1/discovery/consul_kv/dump
--- error_code: 200
--- response_body_unlike
^{}$
yongboy marked this conversation as resolved.
Show resolved Hide resolved



=== TEST 2: test eureka dump_data api
--- yaml_config eval: $::yaml_config
--- request
GET /v1/discovery/eureka/dump
--- error_code: 200
--- response_body_unlike
^{}$
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Y




=== TEST 3: test dns api
--- yaml_config eval: $::yaml_config
--- request
GET /v1/discovery/dns/dump
--- error_code: 404



=== TEST 4: test unconfiged consul_kv dump_data api
--- yaml_config
apisix:
enable_control: true
node_listen: 1984
config_center: yaml
enable_admin: false

discovery:
consul_kv:
servers:
- "http://127.0.0.1:8500"
- "http://127.0.0.1:8600"
#END
--- request
GET /v1/discovery/eureka/dump
--- error_code: 404
4 changes: 2 additions & 2 deletions t/discovery/consul_kv.t
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ run_tests();

__DATA__

=== TEST 1: clean consul kv register nodes
=== TEST 1: prepare consul kv register nodes
--- config
location /consul1 {
rewrite ^/consul1/(.*) /v1/kv/$1 break;
Expand Down Expand Up @@ -289,7 +289,7 @@ GET /hello



=== TEST 7: test register&register nodes
=== TEST 7: test register and unregister nodes
--- yaml_config eval: $::yaml_config
--- apisix_yaml
routes:
Expand Down