diff --git a/community/cases.md b/community/cases.md index 83eac750d7..61bd7fb01b 100644 --- a/community/cases.md +++ b/community/cases.md @@ -60,4 +60,10 @@ * 公司名称： 欢聚时代 * 落地项目： 推荐、直播 * 使用版本： 基于社区版本定制 -* 信息提供者：chenBright \ No newline at end of file +* 信息提供者：chenBright + +## brpc 在 Apache Doris 中的应用 +* 落地项目：Apache Doris +* 使用版本：1.2.0 +* 使用情况：Apache Doris 作为一款 MPP 分析型数据库，其内部节点间使用 Apache Brpc 作为主要 RPC 框架。Brpc 为 Doris 提供了稳定易用的高性能通信机制。并且 BRPC 提供的 bthread，bvar 等基础库，以及各种性能调试工具，也极大的方便了 Doris 的开发和调试工作。 +* 信息提供者：morningman diff --git a/community/oncall.md b/community/oncall.md index ad09f1ac41..ce03176bb2 100644 --- a/community/oncall.md +++ b/community/oncall.md @@ -46,3 +46,4 @@ | 07/25/2022 - 07/31/2022 | 王伟冰 | https://lists.apache.org/thread/83scwkkfxrp6kkkoltbrn1fthfy3w0qz | 08/08/2022 - 08/14/2022 | 何磊 | https://lists.apache.org/thread/jj16rzfh34yrt6o0xqfdz9wtdtzxzswq | 08/15/2022 - 08/21/2022 | 刘帅 | https://lists.apache.org/thread/jp69sm7c8fs3dkdd828qk0fsojqwwz6h +| 09/05/2022 - 09/12/2022 | 王伟冰 | https://lists.apache.org/thread/4jjk2hxw9s2wskccclqb8fvpqxqffnlb diff --git a/docs/cn/client.md b/docs/cn/client.md index 6da1690907..076549fb75 100755 --- a/docs/cn/client.md +++ b/docs/cn/client.md @@ -141,6 +141,32 @@ BNS是百度内常用的命名服务，比如bns://rdev.matrix.all，其中"bns" 如果consul不可访问，服务可自动降级到file naming service获取服务列表。此功能默认关闭，可通过设置-consul\_enable\_degrade\_to\_file\_naming\_service来打开。服务列表文件目录通过-consul \_file\_naming\_service\_dir来设置，使用service-name作为文件名。该文件可通过consul-template生成，里面会保存consul不可用之前最新的下游服务节点。当consul恢复时可自动恢复到consul naming service。 + +### nacos://\ + +NacosNamingService使用[Open-Api](https://nacos.io/zh-cn/docs/open-api.html)定时从nacos获取服务列表。 +NacosNamingService支持[简单鉴权](https://nacos.io/zh-cn/docs/auth.html)。 + +``是一个http uri query，具体参数参见`/nacos/v1/ns/instance/list`文档。 +注意：``需要urlencode。 +``` +nacos://serviceName=test&groupName=g&namespaceId=n&clusters=c&healthyOnly=true +``` + +NacosNamingService拉取列表的时间间隔为`/nacos/v1/ns/instance/list`api返回的`cacheMillis`。 +NacosNamingService只支持整形的权重值。 + +| GFlags | 描述 | 默认值 | +| ---------------------------------- | -------------------------- | ---------------------------- | +| nacos_address | nacos http url | "" | +| nacos_service_discovery_path | nacos服务发现路径 | "/nacos/v1/ns/instance/list" | +| nacos_service_auth_path | nacos登陆路径 | "/nacos/v1/auth/login" | +| nacos_service_timeout_ms | 连接nacos超时时间（毫秒） | 200 | +| nacos_username | 用户名（urlencode编码） | "" | +| nacos_password | 密码（urlencode编码） | "" | +| nacos_load_balancer | nacos集群的负载均衡 | "rr" | + + ### 更多命名服务 用户可以通过实现brpc::NamingService来对接更多命名服务，具体见[这里](https://github.com/brpc/brpc/blob/master/docs/cn/load_balancing.md#%E5%91%BD%E5%90%8D%E6%9C%8D%E5%8A%A1) diff --git a/docs/cn/getting_started.md b/docs/cn/getting_started.md index bf9dca1f93..3afdb56041 100644 --- a/docs/cn/getting_started.md +++ b/docs/cn/getting_started.md @@ -170,6 +170,17 @@ $ sh run_tests.sh ### 使用cmake编译brpc 参考[这里](#使用cmake编译brpc) +### 使用vcpkg编译brpc + +[vcpkg](https://github.com/microsoft/vcpkg) 是一个全平台支持的包管理器，你可以使用以下步骤vcpkg轻松编译brpc: + +```shell +$ git clone https://github.com/microsoft/vcpkg.git +$ ./bootstrap-vcpkg.bat # 使用 powershell +$ ./bootstrap-vcpkg.sh # 使用 bash +$ ./vcpkg install brpc +``` + ## 自己构建依赖的Linux ### 依赖准备 diff --git a/docs/en/client.md b/docs/en/client.md index dac05b41ab..75cb504b03 100644 --- a/docs/en/client.md +++ b/docs/en/client.md @@ -139,6 +139,32 @@ If the server list returned by the consul does not follow [response format](http If consul is not accessible, the naming service can be automatically downgraded to file naming service. This feature is turned off by default and can be turned on by setting -consul\_enable\_degrade\_to\_file\_naming\_service. After downgrading, in the directory specified by -consul\_file\_naming\_service\_dir, the file whose name is the service-name will be used. This file can be generated by the consul-template, which holds the latest server list before the consul is unavailable. The consul naming service is automatically restored when consul is restored. + +### nacos://\ + +NacosNamingService gets a list of servers from nacos periodically by [Open-Api](https://nacos.io/en-us/docs/open-api.html). +NacosNamingService supports [simple authentication](https://nacos.io/en-us/docs/auth.html). + +`` is a http uri query，For more detail, refer to `/nacos/v1/ns/instance/list` api document. +NOTE: `` must be url-encoded. +``` +nacos://serviceName=test&groupName=g&namespaceId=n&clusters=c&healthyOnly=true +``` + +The server list is cached for `cacheMillis` milliseconds as specified in the response of `/nacos/v1/ns/instance/list` api. +NOTE: The value of server weight must be an integer. + +| GFlags | Description | Default value | +| ---------------------------------- | ------------------------------------ | ---------------------------- | +| nacos_address | nacos http url | "" | +| nacos_service_discovery_path | path for discovery | "/nacos/v1/ns/instance/list" | +| nacos_service_auth_path | path for login | "/nacos/v1/auth/login" | +| nacos_service_timeout_ms | timeout for connecting to nacos(ms) | 200 | +| nacos_username | url-encoded username | "" | +| nacos_password | url-encoded password | "" | +| nacos_load_balancer | load balancer for nacos clusters | "rr" | + + ### More naming services User can extend to more naming services by implementing brpc::NamingService, check [this link](https://github.com/brpc/brpc/blob/master/docs/cn/load_balancing.md#%E5%91%BD%E5%90%8D%E6%9C%8D%E5%8A%A1) for details. diff --git a/docs/en/getting_started.md b/docs/en/getting_started.md index 3baf3a67eb..4b4d14f575 100644 --- a/docs/en/getting_started.md +++ b/docs/en/getting_started.md @@ -107,6 +107,18 @@ Examples link brpc statically, if you need to link the shared version, remove `C $ mkdir build && cd build && cmake -DBUILD_UNIT_TESTS=ON .. && make && make test ``` +### Compile brpc with vcpkg + +[vcpkg](https://github.com/microsoft/vcpkg) is a package manager that supports all platforms, +you can use vcpkg to build llvm with the following step: + +```shell +$ git clone https://github.com/microsoft/vcpkg.git +$ ./bootstrap-vcpkg.bat # for powershell +$ ./bootstrap-vcpkg.sh # for bash +$ ./vcpkg install brpc +``` + ## Fedora/CentOS ### Prepare deps diff --git a/src/brpc/global.cpp b/src/brpc/global.cpp index ced8a11cbc..af8dac5c27 100755 --- a/src/brpc/global.cpp +++ b/src/brpc/global.cpp @@ -38,6 +38,7 @@ #include "brpc/policy/remote_file_naming_service.h" #include "brpc/policy/consul_naming_service.h" #include "brpc/policy/discovery_naming_service.h" +#include "brpc/policy/nacos_naming_service.h" // Load Balancers #include "brpc/policy/round_robin_load_balancer.h" @@ -135,6 +136,7 @@ struct GlobalExtensions { RemoteFileNamingService rfns; ConsulNamingService cns; DiscoveryNamingService dcns; + NacosNamingService nns; RoundRobinLoadBalancer rr_lb; WeightedRoundRobinLoadBalancer wrr_lb; @@ -358,6 +360,7 @@ static void GlobalInitializeOrDieImpl() { NamingServiceExtension()->RegisterOrDie("remotefile", &g_ext->rfns); NamingServiceExtension()->RegisterOrDie("consul", &g_ext->cns); NamingServiceExtension()->RegisterOrDie("discovery", &g_ext->dcns); + NamingServiceExtension()->RegisterOrDie("nacos", &g_ext->nns); // Load Balancers LoadBalancerExtension()->RegisterOrDie("rr", &g_ext->rr_lb); diff --git a/src/brpc/periodic_naming_service.cpp b/src/brpc/periodic_naming_service.cpp index e113624199..5e10977997 100644 --- a/src/brpc/periodic_naming_service.cpp +++ b/src/brpc/periodic_naming_service.cpp @@ -29,6 +29,10 @@ DEFINE_int32(ns_access_interval, 5, "Wait so many seconds before next access to naming service"); BRPC_VALIDATE_GFLAG(ns_access_interval, PositiveInteger); +int PeriodicNamingService::GetNamingServiceAccessIntervalMs() const { + return std::max(FLAGS_ns_access_interval, 1) * 1000; +} + int PeriodicNamingService::RunNamingService( const char* service_name, NamingServiceActions* actions) { std::vector servers; @@ -47,7 +51,7 @@ int PeriodicNamingService::RunNamingService( actions->ResetServers(servers); } - if (bthread_usleep(std::max(FLAGS_ns_access_interval, 1) * 1000000L) < 0) { + if (bthread_usleep(GetNamingServiceAccessIntervalMs() * 1000UL) < 0) { if (errno == ESTOP) { RPC_VLOG << "Quit NamingServiceThread=" << bthread_self(); return 0; diff --git a/src/brpc/periodic_naming_service.h b/src/brpc/periodic_naming_service.h index b27033f720..8216ddfdbc 100644 --- a/src/brpc/periodic_naming_service.h +++ b/src/brpc/periodic_naming_service.h @@ -29,6 +29,8 @@ class PeriodicNamingService : public NamingService { virtual int GetServers(const char *service_name, std::vector* servers) = 0; + virtual int GetNamingServiceAccessIntervalMs() const; + int RunNamingService(const char* service_name, NamingServiceActions* actions); }; diff --git a/src/brpc/policy/http_rpc_protocol.cpp b/src/brpc/policy/http_rpc_protocol.cpp index 22b79ca929..7ae08cb70a 100644 --- a/src/brpc/policy/http_rpc_protocol.cpp +++ b/src/brpc/policy/http_rpc_protocol.cpp @@ -1457,7 +1457,7 @@ void ProcessHttpRequest(InputMessageBase *msg) { if (is_grpc_ct) { bool grpc_compressed = false; if (!RemoveGrpcPrefix(&req_body, &grpc_compressed)) { - cntl->SetFailed(ERESPONSE, "Invalid gRPC response"); + cntl->SetFailed(EREQUEST, "Invalid gRPC request"); return; } if (grpc_compressed) { diff --git a/src/brpc/policy/nacos_naming_service.cpp b/src/brpc/policy/nacos_naming_service.cpp new file mode 100644 index 0000000000..0431626ea5 --- /dev/null +++ b/src/brpc/policy/nacos_naming_service.cpp @@ -0,0 +1,289 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include "nacos_naming_service.h" + +#include + +#include + +#include "brpc/http_status_code.h" +#include "brpc/log.h" +#include "butil/iobuf.h" +#include "butil/logging.h" +#include "butil/third_party/rapidjson/document.h" + +namespace brpc { +namespace policy { + +DEFINE_string(nacos_address, "", + "The query string of request nacos for discovering service."); +DEFINE_string(nacos_service_discovery_path, "/nacos/v1/ns/instance/list", + "The url path for discovering service."); +DEFINE_string(nacos_service_auth_path, "/nacos/v1/auth/login", + "The url path for authentiction."); +DEFINE_int32(nacos_connect_timeout_ms, 200, + "Timeout for creating connections to nacos in milliseconds"); +DEFINE_string(nacos_username, "", "nacos username"); +DEFINE_string(nacos_password, "", "nacos password"); +DEFINE_string(nacos_load_balancer, "rr", "nacos load balancer name"); + +int NacosNamingService::Connect() { + ChannelOptions opt; + opt.protocol = PROTOCOL_HTTP; + opt.connect_timeout_ms = FLAGS_nacos_connect_timeout_ms; + const int ret = _channel.Init(FLAGS_nacos_address.c_str(), + FLAGS_nacos_load_balancer.c_str(), &opt); + if (ret != 0) { + LOG(ERROR) << "Fail to init channel to nacos at " + << FLAGS_nacos_address; + } + return ret; +} + +int NacosNamingService::RefreshAccessToken(const char *service_name) { + Controller cntl; + cntl.http_request().uri() = FLAGS_nacos_service_auth_path; + cntl.http_request().set_method(brpc::HttpMethod::HTTP_METHOD_POST); + cntl.http_request().set_content_type("application/x-www-form-urlencoded"); + + auto &buf = cntl.request_attachment(); + buf.append("username="); + buf.append(FLAGS_nacos_username); + buf.append("&password="); + buf.append(FLAGS_nacos_password); + + _channel.CallMethod(nullptr, &cntl, nullptr, nullptr, nullptr); + if (cntl.Failed()) { + LOG(ERROR) << "Fail to access " << FLAGS_nacos_service_auth_path << ": " + << cntl.ErrorText(); + return -1; + } + + BUTIL_RAPIDJSON_NAMESPACE::Document doc; + if (doc.Parse(cntl.response_attachment().to_string().c_str()) + .HasParseError()) { + LOG(ERROR) << "Failed to parse nacos auth response"; + return -1; + } + if (!doc.IsObject()) { + LOG(ERROR) << "The nacos's auth response for " << service_name + << " is not a json object"; + return -1; + } + + auto iter = doc.FindMember("accessToken"); + if (iter != doc.MemberEnd() && iter->value.IsString()) { + _access_token = iter->value.GetString(); + } else { + LOG(ERROR) << "The nacos's auth response for " << service_name + << " has no accessToken field"; + return -1; + } + + auto iter_ttl = doc.FindMember("tokenTtl"); + if (iter_ttl != doc.MemberEnd() && iter_ttl->value.IsInt()) { + _token_expire_time = time(NULL) + iter_ttl->value.GetInt() - 10; + } else { + _token_expire_time = 0; + } + + return 0; +} + +int NacosNamingService::GetServerNodes(const char *service_name, + bool token_changed, + std::vector *nodes) { + if (_nacos_url.empty() || token_changed) { + _nacos_url = FLAGS_nacos_service_discovery_path; + _nacos_url += "?"; + if (!_access_token.empty()) { + _nacos_url += "accessToken=" + _access_token; + _nacos_url += "&"; + } + _nacos_url += service_name; + } + + Controller cntl; + cntl.http_request().uri() = _nacos_url; + _channel.CallMethod(nullptr, &cntl, nullptr, nullptr, nullptr); + if (cntl.Failed()) { + LOG(ERROR) << "Fail to access " << _nacos_url << ": " + << cntl.ErrorText(); + return -1; + } + if (cntl.http_response().status_code() != HTTP_STATUS_OK) { + LOG(ERROR) << "Failed to request nacos, http status code: " + << cntl.http_response().status_code(); + return -1; + } + + BUTIL_RAPIDJSON_NAMESPACE::Document doc; + if (doc.Parse(cntl.response_attachment().to_string().c_str()) + .HasParseError()) { + LOG(ERROR) << "Failed to parse nacos response"; + return -1; + } + if (!doc.IsObject()) { + LOG(ERROR) << "The nacos's response for " << service_name + << " is not a json object"; + return -1; + } + + auto it_hosts = doc.FindMember("hosts"); + if (it_hosts == doc.MemberEnd()) { + LOG(ERROR) << "The nacos's response for " << service_name + << " has no hosts member"; + return -1; + } + auto &hosts = it_hosts->value; + if (!hosts.IsArray()) { + LOG(ERROR) << "hosts member in nacos response is not an array"; + return -1; + } + + std::set presence; + for (auto it = hosts.Begin(); it != hosts.End(); ++it) { + auto &host = *it; + if (!host.IsObject()) { + LOG(ERROR) << "host member in nacos response is not an object"; + continue; + } + + auto it_ip = host.FindMember("ip"); + if (it_ip == host.MemberEnd() || !it_ip->value.IsString()) { + LOG(ERROR) << "host in nacos response has not ip"; + continue; + } + auto &ip = it_ip->value; + + auto it_port = host.FindMember("port"); + if (it_port == host.MemberEnd() || !it_port->value.IsInt()) { + LOG(ERROR) << "host in nacos response has not port"; + continue; + } + auto &port = it_port->value; + + auto it_enabled = host.FindMember("enabled"); + if (it_enabled == host.MemberEnd() || !(it_enabled->value.IsBool()) || + !(it_enabled->value.GetBool())) { + LOG(INFO) << "nacos " << ip.GetString() << ":" << port.GetInt() + << " is not enable"; + continue; + } + + auto it_healthy = host.FindMember("healthy"); + if (it_healthy == host.MemberEnd() || !(it_healthy->value.IsBool()) || + !(it_healthy->value.GetBool())) { + LOG(INFO) << "nacos " << ip.GetString() << ":" << port.GetInt() + << " is not healthy"; + continue; + } + + butil::EndPoint end_point; + if (str2endpoint(ip.GetString(), port.GetUint(), &end_point) != 0) { + LOG(ERROR) << "ncos service with illegal address or port: " + << ip.GetString() << ":" << port.GetUint(); + continue; + } + + ServerNode node(end_point); + auto it_weight = host.FindMember("weight"); + if (it_weight != host.MemberEnd() && it_weight->value.IsNumber()) { + node.tag = + std::to_string(static_cast(it_weight->value.GetDouble())); + } + + presence.insert(node); + } + + nodes->reserve(presence.size()); + nodes->assign(presence.begin(), presence.end()); + + if (nodes->empty() && hosts.Size() != 0) { + LOG(ERROR) << "All service about " << service_name + << " from nacos is invalid, refuse to update servers"; + return -1; + } + + RPC_VLOG << "Got " << nodes->size() + << (nodes->size() > 1 ? " servers" : " server") << " from " + << service_name; + + auto it_cache = doc.FindMember("cacheMillis"); + if (it_cache != doc.MemberEnd() && it_cache->value.IsInt64()) { + _cache_ms = it_cache->value.GetInt64(); + } + + return 0; +} + +NacosNamingService::NacosNamingService() + : _nacos_connected(false), _cache_ms(-1), _token_expire_time(0) {} + +int NacosNamingService::GetNamingServiceAccessIntervalMs() const { + if (0 < _cache_ms) { + return _cache_ms; + } + return PeriodicNamingService::GetNamingServiceAccessIntervalMs(); +} + +int NacosNamingService::GetServers(const char *service_name, + std::vector *servers) { + if (!_nacos_connected) { + const int ret = Connect(); + if (0 == ret) { + _nacos_connected = true; + } else { + return ret; + } + } + + const bool authentiction_enabled = + !FLAGS_nacos_username.empty() && !FLAGS_nacos_password.empty(); + const bool has_invalid_access_token = + _access_token.empty() || + (0 < _token_expire_time && _token_expire_time <= time(NULL)); + bool token_changed = false; + + if (authentiction_enabled && has_invalid_access_token) { + const int ret = RefreshAccessToken(service_name); + if (ret == 0) { + token_changed = true; + } else { + return ret; + } + } + + servers->clear(); + return GetServerNodes(service_name, token_changed, servers); +} + +void NacosNamingService::Describe(std::ostream &os, + const DescribeOptions &) const { + os << "nacos"; + return; +} + +NamingService *NacosNamingService::New() const { + return new NacosNamingService; +} + +void NacosNamingService::Destroy() { delete this; } + +} // namespace policy +} // namespace brpc diff --git a/src/brpc/policy/nacos_naming_service.h b/src/brpc/policy/nacos_naming_service.h new file mode 100644 index 0000000000..dcd7713617 --- /dev/null +++ b/src/brpc/policy/nacos_naming_service.h @@ -0,0 +1,67 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#ifndef BRPC_POLICY_NACOS_NAMING_SERVICE_H +#define BRPC_POLICY_NACOS_NAMING_SERVICE_H + +#include + +#include +#include + +#include "brpc/channel.h" +#include "brpc/periodic_naming_service.h" +#include "brpc/server_node.h" + +namespace brpc { +namespace policy { + +// Acquire server list from nacos +class NacosNamingService : public PeriodicNamingService { +public: + NacosNamingService(); + + int GetServers(const char* service_name, + std::vector* servers) override; + + int GetNamingServiceAccessIntervalMs() const override; + + void Describe(std::ostream& os, const DescribeOptions&) const override; + + NamingService* New() const override; + + void Destroy() override; + +private: + int Connect(); + int RefreshAccessToken(const char* service_name); + int GetServerNodes(const char* service_name, bool token_changed, + std::vector* nodes); + +private: + brpc::Channel _channel; + std::string _nacos_url; + std::string _access_token; + bool _nacos_connected; + long _cache_ms; + time_t _token_expire_time; +}; + +} // namespace policy +} // namespace brpc + +#endif // BRPC_POLICY_NACOS_NAMING_SERVICE_H diff --git a/src/butil/errno.cpp b/src/butil/errno.cpp index 8b3eeb7604..9b964e114f 100644 --- a/src/butil/errno.cpp +++ b/src/butil/errno.cpp @@ -60,9 +60,8 @@ int DescribeCustomizedErrno( if (desc && strncmp(desc, "Unknown error", 13) != 0) #endif { - fprintf(stderr, "Fail to define %s(%d) which is already defined as `%s', abort.", + fprintf(stderr, "WARNING: Fail to define %s(%d) which is already defined as `%s'", error_name, error_code, desc); - _exit(1); } } errno_desc[error_code - ERRNO_BEGIN] = description; diff --git a/src/bvar/variable.cpp b/src/bvar/variable.cpp index c14e8dcb12..7c7f523b3e 100644 --- a/src/bvar/variable.cpp +++ b/src/bvar/variable.cpp @@ -705,9 +705,9 @@ DEFINE_string(bvar_dump_exclude, "", "Dump bvar excluded from these wildcards, " "separated by semicolon(;), empty means no exclusion"); DEFINE_string(bvar_dump_prefix, "", "Every dumped name starts with this prefix"); DEFINE_string(bvar_dump_tabs, "latency=*_latency*" - "; qps=*_qps*" - "; error=*_error*" - "; system=*process_*,*malloc_*,*kernel_*", + ";qps=*_qps*" + ";error=*_error*" + ";system=*process_*,*malloc_*,*kernel_*", "Dump bvar into different tabs according to the filters (seperated by semicolon), " "format: *(tab_name=wildcards;)"); diff --git a/test/brpc_naming_service_unittest.cpp b/test/brpc_naming_service_unittest.cpp index e6a62c85d6..43ac9f474a 100644 --- a/test/brpc_naming_service_unittest.cpp +++ b/test/brpc_naming_service_unittest.cpp @@ -19,8 +19,10 @@ #include #include #include "butil/string_printf.h" +#include "butil/strings/string_split.h" #include "butil/files/temp_file.h" #include "bthread/bthread.h" +#include "brpc/http_status_code.h" #ifdef BAIDU_INTERNAL #include "brpc/policy/baidu_naming_service.h" #endif @@ -30,6 +32,7 @@ #include "brpc/policy/list_naming_service.h" #include "brpc/policy/remote_file_naming_service.h" #include "brpc/policy/discovery_naming_service.h" +#include "brpc/policy/nacos_naming_service.h" #include "echo.pb.h" #include "brpc/server.h" @@ -45,6 +48,9 @@ DECLARE_string(consul_service_discovery_url); DECLARE_string(discovery_api_addr); DECLARE_string(discovery_env); DECLARE_int32(discovery_renew_interval_s); +DECLARE_string(nacos_address); +DECLARE_string(nacos_username); +DECLARE_string(nacos_password); } // policy } // brpc @@ -697,4 +703,139 @@ TEST(NamingServiceTest, discovery_sanity) { } } +class NacosNamingServiceImpl : public test::NacosNamingService { +public: + void Login(google::protobuf::RpcController* cntl_base, + const test::HttpRequest*, test::HttpResponse*, + google::protobuf::Closure* done) override { + brpc::ClosureGuard done_guard(done); + brpc::Controller* cntl = static_cast(cntl_base); + + butil::StringPairs user; + butil::SplitStringIntoKeyValuePairs( + cntl->request_attachment().to_string(), '=', '&', &user); + + const auto expected_user = + butil::StringPairs{{"username", "nacos"}, {"password", "nacos"}}; + + if (user == expected_user) { + cntl->http_response().set_content_type("application/json"); + cntl->response_attachment().append( +R"({ + "accessToken": "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTY2MzAwODMzNn0.YKJJwzHT4v9cpC7kVqWroeJK1WioOYe0JZy4KX8nExs", + "tokenTtl": 18000, + "globalAdmin": true, + "username": "nacos" + })"); + } else { + cntl->http_response().set_status_code(brpc::HTTP_STATUS_FORBIDDEN); + cntl->response_attachment().append("unknow user!"); + } + } + + void List(google::protobuf::RpcController* cntl_base, + const test::HttpRequest*, test::HttpResponse*, + google::protobuf::Closure* done) override { + brpc::ClosureGuard done_guard(done); + brpc::Controller* cntl = (brpc::Controller*)cntl_base; + + auto token = cntl->http_request().uri().GetQuery("accessToken"); + if (token == nullptr || + *token != + "eyJhbGciOiJIUzI1NiJ9." + "eyJzdWIiOiJuYWNvcyIsImV4cCI6MTY2MzAwODMzNn0." + "YKJJwzHT4v9cpC7kVqWroeJK1WioOYe0JZy4KX8nExs") { + cntl->http_response().set_status_code(brpc::HTTP_STATUS_FORBIDDEN); + cntl->response_attachment().append( +R"({ + "timestamp": "2022-09-12T22:56:02.730+08:00", + "status": 403, + "error": "Forbidden", + "path": "/nacos/v1/ns/instance/list" + })"); + return; + } + + auto service_name = cntl->http_request().uri().GetQuery("serviceName"); + auto group_name = cntl->http_request().uri().GetQuery("groupName"); + auto namespace_id = cntl->http_request().uri().GetQuery("namespaceId"); + auto clusters = cntl->http_request().uri().GetQuery("clusters"); + if (service_name == nullptr || *service_name != "test" || + group_name == nullptr || *group_name != "g1" || + namespace_id == nullptr || *namespace_id != "n1" || + clusters == nullptr || *clusters != "wx") { + cntl->http_response().set_status_code(brpc::HTTP_STATUS_NOT_FOUND); + return; + } + + cntl->http_response().set_content_type("application/json"); + cntl->response_attachment().append( +R"({ + "name": "g1@@test", + "groupName": "g1", + "clusters": "wx", + "cacheMillis": 10000, + "hosts": + [ + { + "instanceId": "127.0.0.1#8888#wx#g1@@test", + "ip": "127.0.0.1", + "port": 8888, + "weight": 10.0, + "healthy": true, + "enabled": true, + "ephemeral": true, + "clusterName": "wx", + "serviceName": "g1@@test", + "metadata": {}, + "instanceHeartBeatInterval": 5000, + "instanceHeartBeatTimeOut": 15000, + "ipDeleteTimeout": 30000, + "instanceIdGenerator": "simple" + } + ], + "lastRefTime": 1662990336712, + "checksum": "", + "allIPs": false, + "reachProtectionThreshold": false, + "valid": true + })"); + } +}; + +TEST(NamingServiceTest, nacos) { + brpc::Server server; + NacosNamingServiceImpl svc; + ASSERT_EQ(0, server.AddService(&svc, brpc::SERVER_DOESNT_OWN_SERVICE, + "/nacos/v1/auth/login => Login, " + "/nacos/v1/ns/instance/list => List")); + ASSERT_EQ(0, server.Start("localhost:8848", nullptr)); + + bthread_usleep(5000000); + + butil::EndPoint ep; + ASSERT_EQ(0, butil::str2endpoint("127.0.0.1:8888", &ep)); + const auto expected_node = brpc::ServerNode(ep, "10"); + + const char* service_name = + "serviceName=test&groupName=g1&namespaceId=n1&clusters=wx"; + brpc::policy::FLAGS_nacos_address = "http://localhost:8848"; + brpc::policy::FLAGS_nacos_username = "nacos"; + brpc::policy::FLAGS_nacos_password = "nacos"; + + { + brpc::policy::NacosNamingService nns; + std::vector nodes; + ASSERT_EQ(0, nns.GetServers(service_name, &nodes)); + ASSERT_EQ(nodes.size(), 1); + ASSERT_EQ(expected_node, nodes[0]); + } + { + brpc::policy::FLAGS_nacos_password = "invalid_password"; + brpc::policy::NacosNamingService nns; + std::vector nodes; + ASSERT_NE(0, nns.GetServers(service_name, &nodes)); + } +} + } //namespace diff --git a/test/echo.proto b/test/echo.proto index 10e12d474c..2a47b234e9 100644 --- a/test/echo.proto +++ b/test/echo.proto @@ -78,6 +78,11 @@ service DiscoveryNamingService { rpc Cancel(HttpRequest) returns (HttpResponse); }; +service NacosNamingService { + rpc Login(HttpRequest) returns (HttpResponse); + rpc List(HttpRequest) returns (HttpResponse); +}; + enum State0 { STATE0_NUM_0 = 0; STATE0_NUM_1 = 1;