LB; improve PMTUd support for external clients

[zolug]

Description

In LB-FE make sure to return ICMP reply to external party from the VIP src the offending packet was sent to.

This is achieved by using nftables rules doing stateless SNAT on ICMP replies generated by the LB-FE.
Only replies triggered by incoming packets with VIP dst are of interest. (In which case MTU related ICMP reply can be triggered during forwarding of the original packet.)

Solution requires sysctl fwmark_reflect enabled for both ipv4 and ipv6:

• Helps limit scope of the introduced rules by checking if fwmark is non-zero.
• Inheriting the fwmark value of the original packet allows for an early successful route lookup due to the Target specific routes in place, thus the ICMP reply can be generated by the stack (instead of being discarded due to unreachable error).
  https://elixir.bootlin.com/linux/v5.10/source/net/ipv6/icmp.c#L598
  https://elixir.bootlin.com/linux/v5.10.194/source/net/ipv4/icmp.c#L744

In case using fwmark_reflect sysctls is not desirable:

If it could be granted that "plain routes" even pointing to wrong network (e.g. default routes for both IPv4 and IPv6) would exist for all possible remote addresses for which an ICMP Frag Needed/Packet Too Big should be returned, then nft rules could be replaced to check the encapsulated original packets addresses instead of relying on fwmark value. (Orig dst must be a VIP, but not the src.)

(In case of IPv6 adding a src route to match packets from IPv6 address of the external interface and point them e.g. towards the BIRD maintained routing table also works.)

Test:

• Lower Meridio internal MTU by setting the NSM_MTU env variable in Proxy (e.g. to 1280)
• Setup a Kind cluster with Meridio and external-host. Running 1 LB-FE is sufficient
• Run tcpdump in both LB-FE and external-host capturing ICMP Frag Needed and ICMPv6 Packet Too Big
• On external-host use ping to send out big packets (use "-M do" in case of IPv4 to force IP DF bit):
  ping 20.0.0.1 -s 1380 -M do
  ping 2000::1 -s 1380 -M do
  Additionally, it's advised to configure additional new IPv4 and IPv6 "TG" addresses on the external-host's vlan0 interface, and use those to send out pings from (to not rely on existing device routes in the LB-FE):
  ping -I 5.5.5.5 20.0.0.1 -s 1380 -M do
  ping -I 5000::1 2000::1 -s 1380 -M do
• The ICMP replies must be sent out by the LB-FE using the VIP address as source the offending packet was destined to.
  Try without the PR as well, in which case depending on the src address of the original packet the ICMP reply will be either sent out to the primary network, or the external network using an address configured on the external interface.

Issue link

#452

Checklist

• Purpose
  • Bug fix
  • New functionality
  • Documentation
  • Refactoring
  • CI
• Test
  • Unit test
  • E2E Test
  • Tested manually
• Introduce a breaking change
  • Yes (description required)
  • No

[zolug]

/reverify