Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ovn lb select the local chassis's backend prefer #4894

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

changluyi
Copy link
Collaborator

@changluyi changluyi commented Jan 2, 2025

Pull Request

What type of this PR

Examples of user facing changes:

  • Features
  • Bug fixes
  • Docs
  • Tests

ovn 实现原理。

OVN 这部分代码修改原理:

实现原理是获取到vip backend的 chassis , 新增加流表入下:

is_chassis_resident("nginx-6f55f97f94-8kvlx.default") , 其中 nginx-6f55f97f94-8kvlx.default 是某个vip 的backend下的任意一个lsp,backends=10.16.0.3:80 表示这个lb 在这个chassis下的所有backendip: port 。

  table=13(ls_in_lb           ), priority=140  , match=(ct.new && ip4.dst == 1.1.1.1 && tcp.dst == 80 && is_chassis_resident("nginx-6f55f97f94-8kvlx.default")), action=(ct_lb_mark(backends=10.16.0.3:80; hash_fields="ip_src");)
  table=13(ls_in_lb           ), priority=140  , match=(ct.new && ip4.dst == 1.1.1.1 && tcp.dst == 80 && is_chassis_resident("nginx-6f55f97f94-zlr4p.default")), action=(ct_lb_mark(backends=10.16.0.2:80; hash_fields="ip_src");)

如果hit不到以上流表,就会走原来的ovn lb 规则。

该功能需要配置Load_balancer的 两个字段
option:prefer_local_backend

option:prefer_local_backend=true

ip_port_mappings

"10.16.0.10"="nginx-6f55f97f94-m4bxx.default:10.16.0.4", "10.16.0.2"="nginx-6f55f97f94-zlr4p.default:10.16.0.4", "10.16.0.3"="nginx-6f55f97f94-8kvlx.default:10.16.0.4", "10.16.0.5"="nginx-6f55f97f94-ns9th.default:10.16.0.4"

kube-ovn 增加场景

场景1:switchlbRules 支持 OVN LB 优先走本地 backend

使用方法:

kube-ovn-controller 开启开关 --enable-ovn-lb-prefer-local = true

用例:

kind: SwitchLBRule
metadata:
  name:  cjh-slr-nginx
spec:
  vip: 1.1.1.1
  sessionAffinity: ClientIP
  namespace: default
  selector:
    - app:nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nginx
              topologyKey: "kubernetes.io/hostname"
      containers:
      - name: nginx-container
        image: nginx:alpine
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80

场景2: ovn lb 模式下支持 internalTrafficPolicy: Local , 真正实现发送本地 backend

使用方法:

kube-ovn-controller 开启开关 --enable-ovn-lb-prefer-local = true

用例:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app-container
        image: nginx:alpine
        ports:
        - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  internalTrafficPolicy: Local
  type: ClusterIP

场景3:underlay 模式下 和 metallb 一起使用, 该场景主要是为了metallb 模式下支持某个underlay subnet提供外部地址池的vip, vip的后端pod同样在该子网下, 并保证clientIP 不被改变。

流量走向:

image

使用方法:

  1. kube-ovn-controller 开启开关 --enable-ovn-lb-prefer-local = true
  2. subnet.spec.enableExternalLBAddress = true
  3. subnet.spec.excludeIps 添加外部地址池的地址段。
  4. service 配置 externalTrafficPolicy:Local

Which issue(s) this PR fixes

Fixes #(issue-number)

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jan 2, 2025
@coveralls
Copy link

coveralls commented Jan 2, 2025

Pull Request Test Coverage Report for Build 13152489564

Details

  • 0 of 445 (0.0%) changed or added relevant lines in 8 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage decreased (-0.2%) to 22.129%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/controller/config.go 0 2 0.0%
pkg/controller/init.go 0 5 0.0%
pkg/ovs/ovn-nb-load_balancer.go 0 32 0.0%
pkg/controller/endpoint.go 0 35 0.0%
pkg/controller/service.go 0 40 0.0%
pkg/daemon/controller.go 0 45 0.0%
pkg/ovs/ovs-ofctl.go 0 117 0.0%
pkg/daemon/controller_linux.go 0 169 0.0%
Files with Coverage Reduction New Missed Lines %
pkg/controller/init.go 1 0.0%
Totals Coverage Status
Change from base Build 13148817566: -0.2%
Covered Lines: 10376
Relevant Lines: 46889

💛 - Coveralls

@changluyi changluyi force-pushed the lb_service_select_local_backend branch from 93b8ed4 to 318cff3 Compare January 6, 2025 03:06
@changluyi changluyi changed the title template test ovn lb select the local chassis's backend prefer Jan 6, 2025
@changluyi changluyi added feature New network feature enhancement Improve exist functions labels Jan 6, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from 318cff3 to cdbf329 Compare January 7, 2025 11:30
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jan 7, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from cdbf329 to ffd7994 Compare January 8, 2025 02:47
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jan 8, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from ffd7994 to 21e47a8 Compare January 9, 2025 05:30
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jan 9, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from 21e47a8 to 2ade8ff Compare January 9, 2025 05:31
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Jan 9, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from 2ade8ff to add2969 Compare January 9, 2025 06:13
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jan 15, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch 2 times, most recently from 8a81fc8 to 4bd7410 Compare January 20, 2025 09:43
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Jan 23, 2025
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from a703a56 to d1385c6 Compare January 24, 2025 02:10
@changluyi changluyi requested a review from zhangzujian January 24, 2025 02:24
e2e.mk Show resolved Hide resolved
e2e.mk Outdated Show resolved Hide resolved
Signed-off-by: clyi <clyi@alauda.io>
Signed-off-by: clyi <clyi@alauda.io>
@changluyi changluyi force-pushed the lb_service_select_local_backend branch 2 times, most recently from 48a02a6 to 692ec96 Compare February 5, 2025 03:06
Signed-off-by: clyi <clyi@alauda.io>
@changluyi changluyi force-pushed the lb_service_select_local_backend branch from 692ec96 to c24b4c3 Compare February 5, 2025 03:30
pkg/controller/service.go Show resolved Hide resolved
pkg/daemon/controller_linux.go Show resolved Hide resolved
Signed-off-by: clyi <clyi@alauda.io>
@zhangzujian
Copy link
Member

The following occasional failure need to be fixed:

  [FAILED] Code Location: /home/zhang/kube-ovn/test/e2e/metallb/e2e_test.go:417
  Expected
      <string>: kube-ovn-control-plane
  to equal
      <string>: kube-ovn-worker
  In [It] at: /home/zhang/kube-ovn/test/e2e/metallb/e2e_test.go:358 @ 02/06/25 02:45:06.75

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improve exist functions feature New network feature size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants