Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kube-ovn-1.12.1 snat no effect #3253

Closed
zbb88888 opened this issue Sep 25, 2023 · 0 comments
Closed

kube-ovn-1.12.1 snat no effect #3253

zbb88888 opened this issue Sep 25, 2023 · 0 comments
Assignees
@zbb88888

Comments

@zbb88888
Copy link
Collaborator

zbb88888 commented Sep 25, 2023 •
edited
Loading

Expected Behavior

kube-ovn-1.12.1 snat has effect

Actual Behavior

kube-ovn-1.12.1 snat has no effect

Steps to Reproduce the Problem

(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get provider-network external -o yaml
apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"kubeovn.io/v1","kind":"ProviderNetwork","metadata":{"annotations":{},"name":"external"},"spec":{"defaultInterface":"enx4ce173470bd6"}}
  creationTimestamp: "2023-09-25T07:27:00Z"
  generation: 1
  name: external
  resourceVersion: "1315"
  uid: 15c61078-61c0-4659-b1da-df377a4919a0
spec:
  defaultInterface: enx4ce173470bd6
status:
  conditions:
  - lastTransitionTime: "2023-09-25T07:27:06Z"
    lastUpdateTime: "2023-09-25T07:27:06Z"
    node: empty
    reason: InitOVSBridgeSucceeded
    status: "True"
    type: Ready
  ready: true
  readyNodes:
  - empty
  vlans:
  - vlan0


(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get vlan -o yaml
apiVersion: v1
items:
- apiVersion: kubeovn.io/v1
  kind: Vlan
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"kubeovn.io/v1","kind":"Vlan","metadata":{"annotations":{},"name":"vlan0"},"spec":{"id":0,"provider":"external"}}
    creationTimestamp: "2023-09-25T07:27:00Z"
    generation: 1
    name: vlan0
    resourceVersion: "1288"
    uid: c67df1c6-c4b2-4cf4-9059-283eb15d56e3
  spec:
    id: 0
    provider: external
  status:
    subnets:
    - external
kind: List
metadata:
  resourceVersion: ""


(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get subnet external -o yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"kubeovn.io/v1","kind":"Subnet","metadata":{"annotations":{},"name":"external"},"spec":{"cidrBlock":"192.168.7.0/24","logicalGateway":true,"protocol":"IPv4","vlan":"vlan0"}}
  creationTimestamp: "2023-09-25T07:27:00Z"
  finalizers:
  - kube-ovn-controller
  generation: 2
  name: external
  resourceVersion: "2044"
  uid: 69321f69-4f03-4af0-886b-1061554e9257
spec:
  cidrBlock: 192.168.7.0/24
  default: false
  enableLb: true
  excludeIps:
  - 192.168.7.1
  gateway: 192.168.7.1
  gatewayNode: ""
  gatewayType: distributed
  logicalGateway: true
  natOutgoing: false
  private: false
  protocol: IPv4
  provider: ovn
  vlan: vlan0
  vpc: ovn-cluster
status:
  activateGateway: ""
  conditions:
  - lastTransitionTime: "2023-09-25T07:27:01Z"
    lastUpdateTime: "2023-09-25T07:28:01Z"
    reason: ResetLogicalSwitchAclSuccess
    status: "True"
    type: Validated
  - lastTransitionTime: "2023-09-25T07:27:01Z"
    lastUpdateTime: "2023-09-25T07:27:01Z"
    reason: ResetLogicalSwitchAclSuccess
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-09-25T07:27:01Z"
    lastUpdateTime: "2023-09-25T07:27:01Z"
    message: Not Observed
    reason: Init
    status: Unknown
    type: Error
  dhcpV4OptionsUUID: ""
  dhcpV6OptionsUUID: ""
  natOutgoingPolicyRules: []
  u2oInterconnectionIP: ""
  u2oInterconnectionVPC: ""
  v4availableIPrange: 192.168.7.4-192.168.7.254
  v4availableIPs: 253
  v4usingIPrange: 192.168.7.2-192.168.7.3
  v4usingIPs: 0
  v6availableIPrange: ""
  v6availableIPs: 0
  v6usingIPrange: ""
  v6usingIPs: 0


(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get vpc
NAME          ENABLEEXTERNAL   ENABLEBFD   STANDBY   SUBNETS                             NAMESPACES
ovn-cluster   true             false       true      ["join","ovn-default","external"]
vpc1          true             false       true      ["vpc1-subnet1"]                    ["vpc1"]
(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get subnet
NAME           PROVIDER   VPC           PROTOCOL   CIDR             PRIVATE   NAT     DEFAULT   GATEWAYTYPE   V4USED   V4AVAILABLE   V6USED   V6AVAILABLE   EXCLUDEIPS        U2OINTERCONNECTIONIP
external       ovn        ovn-cluster   IPv4       192.168.7.0/24   false     false   false     distributed   0        253           0        0             ["192.168.7.1"]
join           ovn        ovn-cluster   IPv4       100.64.0.0/16    false     false   false     distributed   1        65532         0        0             ["100.64.0.1"]
ovn-default    ovn        ovn-cluster   IPv4       10.16.0.0/16     false     true    true      distributed   3        65530         0        0             ["10.16.0.1"]
vpc1-subnet1   ovn        vpc1          IPv4       192.168.0.0/24   false     false   false     distributed   2        251           0        0             ["192.168.0.1"]
(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k get po -A -o wide | grep 192.168
vpc1          vpc-1-busybox01                       1/1     Running   0          8m4s   192.168.0.2      empty   <none>           <none>
vpc1          vpc-1-busybox02                       1/1     Running   0          8m4s   192.168.0.3      empty   <none>           <none>
(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc#


(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc# k ko nbctl show
switch 9e7e94ec-ac80-4dd7-be79-9e7d60ba94d7 (join)
    port node-empty
        addresses: ["00:00:00:A2:61:64 100.64.0.2"]
    port join-ovn-cluster
        type: router
        router-port: ovn-cluster-join
switch 797ec135-98de-418d-a444-6cfed810b4ac (external)
    port external-ovn-cluster
        type: router
        router-port: ovn-cluster-external
    port localnet.external
        type: localnet
        addresses: ["unknown"]
    port external-vpc1
        type: router
        router-port: vpc1-external
switch 03766c41-63aa-42c3-bdf2-ec9503c56adf (vpc1-subnet1)
    port vpc-1-busybox02.vpc1
        addresses: ["00:00:00:2D:A6:6B 192.168.0.3"]
    port vpc-1-busybox01.vpc1
        addresses: ["00:00:00:AD:39:01 192.168.0.2"]
    port vpc1-subnet1-vpc1
        type: router
        router-port: vpc1-vpc1-subnet1
switch da3e0c7e-4dc3-40da-a803-bf8940889c88 (ovn-default)
    port coredns-67ddbf998c-8tmgz.kube-system
        addresses: ["00:00:00:E6:D0:D9 10.16.0.5"]
    port coredns-67ddbf998c-65t8c.kube-system
        addresses: ["00:00:00:A2:78:E7 10.16.0.4"]
    port kube-ovn-pinger-2vwz4.kube-system
        addresses: ["00:00:00:F7:A0:D9 10.16.0.6"]
    port ovn-default-ovn-cluster
        type: router
        router-port: ovn-cluster-ovn-default
router f8a9c629-4988-48f1-8397-ad9ea4b7921b (vpc1)
    port vpc1-external
        mac: "00:00:00:4A:DB:35"
        networks: ["192.168.7.2/24"]
        gateway chassis: [38b5d345-1cc8-492e-b825-215684d08741]
    port vpc1-vpc1-subnet1
        mac: "00:00:00:F0:3F:AE"
        networks: ["192.168.0.1/24"]
    nat 5d30faa3-f0c4-4504-9582-dd6232f444e8
        external ip: "192.168.7.3"
        logical ip: "192.168.0.0/24" # snat to all the subnet 
        type: "snat"
router 4b45d3f2-3e17-476d-979e-7b20cfc15230 (ovn-cluster)
    port ovn-cluster-ovn-default
        mac: "00:00:00:FE:A1:07"
        networks: ["10.16.0.1/16"]
    port ovn-cluster-external
        mac: "00:00:00:8A:8B:5F"
        networks: ["192.168.7.1/24"]
        gateway chassis: [38b5d345-1cc8-492e-b825-215684d08741]
    port ovn-cluster-join
        mac: "00:00:00:B2:F0:95"
        networks: ["100.64.0.1/16"]

########## after i ping the pod to the external network, the source ip is not snated

(.venv) root@empty:~/test/kovn/eip-snat/03-cust-vpc/busybox# k exec -it -n vpc1          vpc-1-busybox01 -- bash
vpc-1-busybox01:~#
vpc-1-busybox01:~#
vpc-1-busybox01:~# ping 192.168.7.200
PING 192.168.7.200 (192.168.7.200) 56(84) bytes of data.
^C
--- 192.168.7.200 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4077ms

vpc-1-busybox01:~# ping 192.168.7.1
PING 192.168.7.1 (192.168.7.1) 56(84) bytes of data.



^C
--- 192.168.7.1 ping statistics ---
21 packets transmitted, 0 received, 100% packet loss, time 20480ms

vpc-1-busybox01:~# ping 192.168.7.2
PING 192.168.7.2 (192.168.7.2) 56(84) bytes of data.
64 bytes from 192.168.7.2: icmp_seq=1 ttl=254 time=0.374 ms
64 bytes from 192.168.7.2: icmp_seq=2 ttl=254 time=0.345 ms
64 bytes from 192.168.7.2: icmp_seq=3 ttl=254 time=0.587 ms
^C
--- 192.168.7.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2052ms
rtt min/avg/max/mdev = 0.345/0.435/0.587/0.107 ms
vpc-1-busybox01:~# ping 192.168.7.200
PING 192.168.7.200 (192.168.7.200) 56(84) bytes of data.
^C
--- 192.168.7.200 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1022ms

vpc-1-busybox01:~#
vpc-1-busybox01:~#
vpc-1-busybox01:~#
vpc-1-busybox01:~#
vpc-1-busybox01:~# ping 192.168.7.254
PING 192.168.7.254 (192.168.7.254) 56(84) bytes of data.
^C
--- 192.168.7.254 ping statistics ---
191 packets transmitted, 0 received, 100% packet loss, time 194557ms


##########  show the packets on the node, the source ip is not snated


(.venv) root@empty:~# tcpdump -i any host 192.168.7.254 -netvv
tcpdump: data link type LINUX_SLL2
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
702a325ae152_h P   ifindex 21 00:00:00:ad:39:01 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 17032, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.2 > 192.168.7.254: ICMP echo request, id 60647, seq 10, length 64
702a325ae152_h P   ifindex 21 00:00:00:ad:39:01 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 17113, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.2 > 192.168.7.254: ICMP echo request, id 60647, seq 11, length 64
702a325ae152_h P   ifindex 21 00:00:00:ad:39:01 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 17332, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.2 > 192.168.7.254: ICMP echo request, id 60647, seq 12, length 64
702a325ae152_h P   ifindex 21 00:00:00:ad:39:01 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 17418, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.2 > 192.168.7.254: ICMP echo request, id 60647, seq 13, length 64
702a325ae152_h P   ifindex 21 00:00:00:ad:39:01 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 17508, offset 0, flags [DF], proto ICMP (1), length 84)

and the one external ip map to pod ip snat is no effect too!

image

(.venv) root@empty:~# tcpdump -i any host 192.168.7.254 -netvv
tcpdump: data link type LINUX_SLL2
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
827171179cb0_h P   ifindex 27 00:00:00:89:b5:46 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 14242, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.3 > 192.168.7.254: ICMP echo request, id 27692, seq 1, length 64
827171179cb0_h P   ifindex 27 00:00:00:89:b5:46 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 14443, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.3 > 192.168.7.254: ICMP echo request, id 27692, seq 2, length 64
827171179cb0_h P   ifindex 27 00:00:00:89:b5:46 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 14612, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.3 > 192.168.7.254: ICMP echo request, id 27692, seq 3, length 64
827171179cb0_h P   ifindex 27 00:00:00:89:b5:46 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 14653, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.3 > 192.168.7.254: ICMP echo request, id 27692, seq 4, length 64
827171179cb0_h P   ifindex 27 00:00:00:89:b5:46 ethertype IPv4 (0x0800), length 104: (tos 0x0, ttl 64, id 14780, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.0.3 > 192.168.7.254: ICMP echo request, id 27692, seq 5, length 64
^C

Additional Info

  • Kubernetes version:

    Output of kubectl version:

    (paste your output here)

  • kube-ovn version:

    (paste your output here)

  • operation-system/kernel version:

    Output of awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release:
    Output of uname -r:

    (paste your output here)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zbb88888 zbb88888

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@zbb88888