v0.4.0
2024 Mid-Year Review: We are pleased to announce release v0.4.0 after more than two months's hard work from all our contributors. In release v0.4.0 we have done many enhancements such as refined pod management, management of DNS typed services, IPv6 protocol support, and data plane communication metrics monitoring. Additionally, improvements have been made in performance, maintainability, and testability, making Kmesh more robust. The key features included in this release are listed but not limited to the following:
Fine-Grained Namespace and Pod Manage:
In addition to namespace-level management, Kmesh now supports fine-grained pod-level management. You can manage pods using the following commands:
# Particular pod manage
kubectl label pod <podName> istio.io/dataplane-mode=kmesh -n {namespace}
# Namespace scoped pods manage
kubectl label ns <namespace> istio.io/dataplane-mode=kmesh
Supports IPv6 Communication in Workload Mode:
Kmesh now supports IPv6 communication, catering to both public and private cloud environments. Even for IPv4 services, Java-based services default to using the IPv6 protocol family. This enhancement ensures broader service management scenarios.
The ads mode will also support IPv6 in the near future, and also we will make Kmesh dual-stack compatible.
Performance Optimization for Rule Refresh in ads Mode:
Previously in the ads mode, Kmesh suffered from slow rule refresh because of map-in-map model used. This release, we significantly improves rule refresh performance by orders of magnitude, laying the groundwork for large-scale cluster management.
Fine-Grained Waypoint Traffic Capture:
To stay in sync with Istio 1.22, Kmesh now supports the latest workload API model. You can now use the “istio.io/use-waypoint” label to enable waypoint capture at the namespace, service, or pod level. This flexibility allows precise and on-demand use of waypoints.
DNS-Type Services Support:
In Kmesh’s ads mode, HTTP protocol is managed using eBPF and kernel modules, along with Listener, Route, and Cluster APIs. However, DNS-typed services posed a challenge because their clusters use domain names as endpoints, making DNS resolution during load balancing impossible within eBPF.
Kmesh introduces a DNS resolve module within the Kmesh daemon. It resolves domain names in user space and rewrites clusters's inlined endpoints to prevent resolving DNS early. As a result, Kmesh now supports Kubernetes ExternalName Services and Istio DNS Resolution ServiceEntry.
Observability Support:
Kmesh prioritizes observability as a critical feature in traffic management. It achieves low-cost connection monitoring using eBPF in the kernel and supports Prometheus-based metrics collection and aggregation. Future updates will enhance other observability capabilities, accesslog and traces.
To query monitoring information, use the following command:
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- curl http://127.0.0.1:15020/status/metrics
Dynamic Log Level Adjustment:
Kmesh now allows dynamic adjustment of log levels for both the kmesh-daemon and eBPF prog. Now kmesh also redirect eBPF data plane logs to user space printing to avoid trace-pipe cost (requires kernel version 5.13 or higher).
# Adjust kmesh-daemon log level (e.g., debug | error | info)
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- kmesh-daemon log --set default:debug
# Adjust kmesh eBPF data plane log level
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- kmesh-daemon log --set bpf:debug
Additionally, there are many other improvements are there:
- Kmesh has introduced an E2E testing framework, ensuring that each PR submission undergoes E2E tests to prevent regressions.
- Reliability reconnections with Istiod have been addressed, and community documentation has been enriched.
What's Changed
- Update Readme by @hzxuzhonghu in #240
- Delete unnecessary code and configuration by @lec-bit in #262
- Fix dns typed service panic by @hzxuzhonghu in #263
- Enable xdp auth when kmesh works in workload mode by @hzxuzhonghu in #260
- Enable dependa bot by @hzxuzhonghu in #231
- Added benchmark test for cluster.Flush() by @LiZhenCheng9527 in #259
- Added benchmark test for listener.Flush() by @LiZhenCheng9527 in #253
- fix authz on server listening on both ipv4 and ipv6 port by @hzxuzhonghu in #272
- opt cluster ut test by @LiZhenCheng9527 in #267
- add yaml of sample applications by @LiZhenCheng9527 in #276
- rename service_id to upstream_id in frontend_value struct by @kwb0523 in #278
- add ut of client.go by @LiZhenCheng9527 in #213
- add clang-format by @hzxuzhonghu in #266
- bump istio to fix proto conflict by @hzxuzhonghu in #284
- opt readme by @LiZhenCheng9527 in #290
- remove duplicate NODE_NAME env by @hzxuzhonghu in #280
- Ignore expected grpc error by @hzxuzhonghu in #283
- Enable merge queue by @hzxuzhonghu in #293
- FIX bpf_map_delete_elem unexpected error log by @hzxuzhonghu in #287
- fix clang format tool not install by @LiZhenCheng9527 in #299
- Kmesh: repair kmesh manager not effect in sockops on oe 23.03 by @bitcoffeeiux in #301
- Refactor: remove global package vars by @hzxuzhonghu in #296
- Fix the issue of incorrect domain matching by @lec-bit in #286
- remove unnecessary port fields in frontend_key by @kwb0523 in #300
- Add proposal template by @hzxuzhonghu in #302
- Fix the Enhanced Compilation Failure Issue by @lec-bit in #309
- apply reviewers and approvers by @supercharge-xsy in #307
- Currently supporting TCP protocol, add the corresponding checks by @supercharge-xsy in #311
- Fix the Enhanced Compilation Failure Issue by @bfforever in #312
- add benchmark of workload by @LiZhenCheng9527 in #316
- optimization: Accelerates the performance of updating the xDS configuration by @nlgwcy in #326
- add optional of start bypass by @LiZhenCheng9527 in #306
- update listener benchmark by @LiZhenCheng9527 in #328
- Update Readme by @hzxuzhonghu in #327
- only update by_key policy_store for workloadp scope authz by @supercharge-xsy in #333
- chore: add kmesh-system namespace api object in kmesh.yaml by @Okabe-Rintarou-0 in #334
- fix bpf-map test failed by @nlgwcy in #341
- Split the ebpf code directory of ads and workload. by @bfforever in #343
- deserialization module optimization by @nlgwcy in #331
- fix: should get authz policy from dst-worklod when rbac by @supercharge-xsy in #336
- Fix some typo errors of some documents by @Okabe-Rintarou-0 in #345
- update workload.proto by @kwb0523 in #346
- use ztunnel-role with workload mode, fix authz policy not work when update by @supercharge-xsy in #344
- docs: translate xds_handle.md to Chinese by @Okabe-Rintarou-0 in #354
- fix: fix issue template by @Okabe-Rintarou-0 in #357
- feat: support kmesh daemon version command(#198) by @Okabe-Rintarou-0 in #356
- fix: make clean does not clean up completely(#264) by @Okabe-Rintarou-0 in #338
- Added dns resolver proposal by @hzxuzhonghu in #305
- Certificate subscription management module to kmesh by @lec-bit in #310
- Monitoring code for UT coverage in github CI using codecov by @LiZhenCheng9527 in #360
- add ut of workload_stream.go by @LiZhenCheng9527 in #204
- add ut of ads_loader.go by @LiZhenCheng9527 in #183
- Improve bpf log and bugs fix by @bfforever in #364
- Fix status server panic by @hzxuzhonghu in #350
- Fix cleanup kmesh-cni failed bug by @bfforever in #367
- add bypass logic for kmesh by @weli-l in #297
- fix: make clean error(#376) by @Okabe-Rintarou-0 in #378
- some workload mode bugfix and waypoint optimization by @kwb0523 in #361
- Clean kmesh build docker container in make clean by @Okabe-Rintarou-0 in #383
- docs: optimize deploy/README.md by @Okabe-Rintarou-0 in #386
- Added instructions on how to become a member of the Kmesh community to the readme. by @LiZhenCheng9527 in #381
- docs: support kmesh tls cert manager by @lec-bit in #379
- fix picture miss by @LiZhenCheng9527 in #389
- optimization for kmesh build slow by @bfforever in #382
- Improve isManagedWorkload by @hzxuzhonghu in #368
- Support Profiling(#199) by @Okabe-Rintarou-0 in #385
- update the workload proposl by @kwb0523 in #369
- Add automatically docker image push by @hzxuzhonghu in #396
- Enable metadata encoder only for waypoint by @hzxuzhonghu in #353
- add proposal for l4 authz by @supercharge-xsy in #391
- Fix build docker image in github action by @hzxuzhonghu in #398
- unified handing logic for skops->remote_port/local_port by @nlgwcy in #395
- Fix dockerimage binary size zero caused by permission by @hzxuzhonghu in #404
- support identity rbac by @supercharge-xsy in #400
- bpf: Dual-license the code as GPL-2.0 OR BSD-2-Clause by @LiZhenCheng9527 in #410
- Dump workload by @hzxuzhonghu in #392
- Make use of FIFQ instead of priority queue by @hzxuzhonghu in #390
- update make clean by @LiZhenCheng9527 in #412
- use peer_metadata to get pod metadata info from istiod by @YaoZengzeng in #411
- Support service dump by @hzxuzhonghu in #413
- Fix bug when delete waypoint, dont update bpf kmesh_service map by @bfforever in #422
- Improve security manager by @hzxuzhonghu in #416
- add make test for run ut easily by @LiZhenCheng9527 in #426
- Added dataplane mode constants by @hzxuzhonghu in #419
- add proposal of Kmesh observability by @LiZhenCheng9527 in #337
- Subscribe cached resources when delta xds reconnect by @Okabe-Rintarou-0 in #429
- Added workload response handler tests by @hzxuzhonghu in #425
- Ipv6 Support - IPv6 compatibility refactor by @nlgwcy in #434
- Provides a solution for running ut with docker and a solution for running locally by @LiZhenCheng9527 in #433
- fix error link in readme by @LiZhenCheng9527 in #441
- Mount proc by @hzxuzhonghu in #432
- fix typos of dns resolver proposal by @YaoZengzeng in #439
- Fix authPolicy remove by @hzxuzhonghu in #435
- add kmesh-daemon dump command by @Okabe-Rintarou-0 in #442
- Bump istio by @hzxuzhonghu in #445
- Update dependabot.yml by @hzxuzhonghu in #444
- Bump google.golang.org/protobuf from 1.34.1 to 1.34.2 by @dependabot in #450
- Bump github.com/containernetworking/plugins from 1.5.0 to 1.5.1 by @dependabot in #448
- Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 by @dependabot in #449
- Enable go race detect by @hzxuzhonghu in #431
- Kmesh eBPF log user space dumping by @bfforever in #402
- Bump github.com/cilium/ebpf from 0.13.2 to 0.15.0 by @dependabot in #451
- Bump the k8s-io group with 3 updates by @dependabot in #447
- Support dynamic changing log level in status manager by @Okabe-Rintarou-0 in #446
- Bug fix on xds handler by @hzxuzhonghu in #438
- Bump github.com/containernetworking/cni from 1.2.0 to 1.2.1 by @dependabot in #457
- Bump istio.io/api from 1.22.0-alpha.1.0.20240612140428-341dd53c476f to 1.22.1 by @dependabot in #458
- bug fix by @nlgwcy in #456
- Simplify byteorder transformation & remove magic number in Authz by @tacslon in #407
- fix compile failed on openEuler 2203 by @nlgwcy in #461
- Refactor clusterCache flush and Fix EDS ACK by @hzxuzhonghu in #454
- Add log sub command for kmesh-daemon by @Okabe-Rintarou-0 in #459
- Bump github.com/containernetworking/cni from 1.2.1 to 1.2.2 by @dependabot in #465
- E2E test framework by @YaoZengzeng in #443
- add security manager test by @lec-bit in #393
- support ipv6 in workload mode by @nlgwcy in #462
- Should send initial authz resources when reconnect by @hzxuzhonghu in #464
- bug fix by @nlgwcy in #473
- Bump google.golang.org/grpc from 1.64.0 to 1.65.0 by @dependabot in #476
- Split E2E test as a separate CI jobs by @LiZhenCheng9527 in #472
- support kmesh manage in pod level by @weli-l in #414
- fix display error of e2e flags by @YaoZengzeng in #481
- optimization: normalized ADDR type by @nlgwcy in #468
- add UT for bypass by @weli-l in #430
- Fix authz goroutine not started by @tacslon in #485
- cleanup unused codes by @hzxuzhonghu in #487
- fix hash name bug (#479) by @Okabe-Rintarou-0 in #482
- Support dns resolution typed service by @PerforMance308 in #436
- Fix bpfloglevel update error bug in Ads mode. by @bfforever in #483
- support run specified unit test (#488) by @Okabe-Rintarou-0 in #491
- merge log handlers (#498) by @Okabe-Rintarou-0 in #500
- fix flaky e2e test by @YaoZengzeng in #499
- keep the same as helm deploy by @Okabe-Rintarou-0 in #513
- Add Okabe-Rintarou-0 to pkg/status owners by @hzxuzhonghu in #502
- update issue label by @hzxuzhonghu in #493
- Bump github.com/miekg/dns from 1.1.59 to 1.1.61 by @dependabot in #505
- Bump github.com/agiledragon/gomonkey/v2 from 2.11.0 to 2.12.0 by @dependabot in #504
- fix workloadController.run() not called error by @LiZhenCheng9527 in #516
- Add some uts for setting bpf log level by @Okabe-Rintarou-0 in #515
- cleanup stopch(#497) by @Okabe-Rintarou-0 in #506
- modify oncn-mda to be compatible with kernel 6.6 by @weli-l in #508
- adjust bpf map size for large-scale cluster by @nlgwcy in #517
- Add compilation macro for oncn-mda by @weli-l in #518
- Add missing copy right by @hzxuzhonghu in #519
- Clean up eBPF compile alarm & ipv6 bug fix by @nlgwcy in #510
- eBPF observability by @nlgwcy in #466
- Support ns manage after kmesh restart by @weli-l in #526
- waypoint listen both ipv4 and ipv6 by @YaoZengzeng in #530
- [release-0.4] kmesh route samples by @kmesh-bot in #532
- [cherry-pick 0.4] Kmesh observability by @LiZhenCheng9527 in #533
- [release-0.4] fix unexpected log by @kmesh-bot in #538
- [release-0.4] Modifybpf map update to prevent potential bugs by @kmesh-bot in #542
- modify kmesh version to v0.4.0 by @lec-bit in #544
- [release-0.4] Fix TestPodSidecarLabelChangeTriggersAddIptablesAction flake by @kmesh-bot in #543
New Contributors
Full Changelog: v0.3.0...v0.4.0
And thank you all to our contributors in this release.
Feel free to talk with us if you need further assistance or have any other questions! 😊