Prefer hosts iptables program

[ibuildthecloud]

#780
#703

[erikwilson]

I made a quick try with this when the initial issued was filed, looks like klipper-lb still has problems, any iptables running in a pod will also need to be updated to use the host iptables.

[erikwilson]

Error from the klipper-lb pod:

$ kubectl logs pod/svclb-traefik-9drx9 -n kube-system lb-port-80
+ trap exit TERM INT
+ cat /proc/sys/net/ipv4/ip_forward
+ '[' 1 '!=' 1 ]
+ iptables -t nat -I PREROUTING '!' -s 10.43.190.44/32 -p TCP --dport 80 -j DNAT --to 10.43.190.44:80
modprobe: can't change directory to '/lib/modules': No such file or directory
iptables v1.6.2: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

Unfortunately I think we should just refuse to start if using nf_tables for the moment, this is a bit of a mess that iptables has created for the kubernetes community.

[ibuildthecloud]

@erikwilson Yeah, I don't really like this because it means that k3s isn't really self contained anymore. I think we need to package both iptables and detect the correct one. I'll see what is the effort to build the new one.

[erikwilson]

@ibuildthecloud I think this needs some more investigation, it really irks me that nf_tables was shoehorned into iptables. However, on Alpine 3.10 using iptables w/ nf_tables everything seems to work fine on k3s v0.8.1, our bundled iptables should probe for the legacy kernel modules and everything appears to come up okay and is usable.

$ kubectl get pods -A
NAMESPACE     NAME                                 READY   STATUS      RESTARTS   AGE
kube-system   coredns-b7464766c-rdgrk              1/1     Running     0          9m27s
kube-system   helm-install-traefik-8ghhs           0/1     Completed   0          9m27s
kube-system   svclb-traefik-4rgws                  2/2     Running     0          9m14s
kube-system   traefik-5c79b789c5-zchzg             1/1     Running     0          9m14s
openfaas      alertmanager-5b9966c9f-prltc         1/1     Running     0          4m3s
openfaas      basic-auth-plugin-85994747dd-2mr9t   1/1     Running     0          4m3s
openfaas      faas-idler-6568bb4c9b-9b9tm          1/1     Running     2          4m2s
openfaas      gateway-8b874bc55-fmv5p              2/2     Running     0          4m2s
openfaas      nats-d4c9d8d95-k5w45                 1/1     Running     0          4m2s
openfaas      prometheus-75c78cd446-kglmn          1/1     Running     0          4m1s
openfaas      queue-worker-56b64d6848-gbfmn        1/1     Running     0          4m2s

$ iptables-save
# Warning: iptables-legacy tables present, use iptables-legacy-save to see them

Probably not ideal to run both, but could make a iptables based firewall actually firewall correctly, so maybe related to the errors in issues.