[Release 1.22] IPv6 masquerade NAT rules missing in dual-stack installation

[rbrtbnfgl]

Backport for #4683

[mdrahman-suse]

Steps are provided here: #4683 (comment)

Reproduced the issue on k3s v1.22.5+k3s2

• ipv4 ping from pod in src node to dest node

$ kubectl run --rm -it --image=centos bash
[root@bash /]# ping -c 5 <dest_ip>
PING <dest_ip>(<dest_ip>) 56(84) bytes of data.
64 bytes from <dest_ip>: icmp_seq=1 ttl=63 time=0.295 ms
...
--- <dest_ip> ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4096ms
rtt min/avg/max/mdev = 0.215/0.264/0.368/0.060 ms

• tcpdump from dest node verifying packets received as NATed

$ tcpdump -i eth0 -n
23:42:50.471929 IP <src_ip> > <dest_ip>: ICMP echo request, id 19007, seq 1, length 64
23:42:50.471962 IP <dest_ip> > <src_ip>: ICMP echo reply, id 19007, seq 1, length 64

• ipv6 ping from pod in src node to dest node

$ kubectl run --rm -it --image=centos bash
If you don't see a command prompt, try pressing enter.
[root@bash /]# ping -c 5 2600:xxx:xxx..:139e
PING 2600:xxx:xxx..:139e(2600:xxx:xxx..:139e) 56 data bytes
--- 2600:xxx:xxx..:139e ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4082ms

Verified the fix on k3s v1.22.6-rc1+k3s1

• ipv4 ping from pod in src node to dest node

$ kubectl run --rm -it --image=centos bash
[root@bash /]# ping -c 5 <dest_ip>
PING <dest_ip>(<dest_ip>) 56(84) bytes of data.
64 bytes from <dest_ip>: icmp_seq=1 ttl=63 time=0.295 ms
...
--- <dest_ip> ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4096ms
rtt min/avg/max/mdev = 0.215/0.264/0.368/0.060 ms

• tcpdump from dest node verifying packets received as NATed

$ tcpdump -i eth0 -n
23:52:08.784508 IP <src_ip> > <dest_ip>: ICMP echo request, id 57375, seq 5, length 64
23:52:08.784522 IP <dest_ip> > <src_ip>: ICMP echo reply, id 57375, seq 5, length 64

• ipv6 ping from pod in src node to dest node

$ kubectl run --rm -it --image=centos bash
If you don't see a command prompt, try pressing enter.
[root@bash /]# ping -c 5 2600:xxx:xxx..:139e
PING 2600:xxx:xxx..:139e(2600:xxx:xxx..:139e) 56 data bytes
64 bytes from 2600:xxx:xxx..:139e: icmp_seq=1 ttl=63 time=0.481 ms
...
--- 2600:xxx:xxx..:139e ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4073ms
rtt min/avg/max/mdev = 0.244/0.315/0.481/0.086 ms

• tcpdump from dest node verifying packets received as NATed

$ tcpdump -i eth0 -n ip6
23:51:00.742624 IP6 2600:xxx:xxx..:139e > 2600:xxx:xxx..:bba8: ICMP6, echo reply, seq 1, length 64
23:51:01.743184 IP6 2600:xxx:xxx..:bba8 > 2600:xxx:xxx..:139e: ICMP6, echo request, seq 2, length 64