Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Public IP firewalls wrong IPv6 #1191

Closed
OmarElawady opened this issue Feb 17, 2021 · 7 comments
Closed

Public IP firewalls wrong IPv6 #1191

OmarElawady opened this issue Feb 17, 2021 · 7 comments
Assignees
@delandtj @LeeSmet
Labels
type_bug Something isn't working

Comments

@OmarElawady
Copy link
Contributor

I can ping this ZDB from the worker node but it doesn't work on the master. Another ZDB is pingable from both.
Screenshot from 2021-02-17 15-24-33

Master wid: 30564
Worker wid: 30565
@OmarElawady OmarElawady added the type_bug Something isn't working label Feb 17, 2021
@LeeSmet
Copy link
Contributor

LeeSmet commented Feb 17, 2021

Can you give the output of ip a and ip -6 r on the master?

@OmarElawady
Copy link
Contributor Author

This is the output of ip a

k3os-8627 [~]$ ip a                                                                                      
	1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
	    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
	    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
	       valid_lft forever preferred_lft forever
	    inet6 ::1/128 scope host                                                                                                                                                                                       
	       valid_lft forever preferred_lft forever                                                                                                                                                                     
	2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
	    link/ether de:c4:57:eb:9b:87 brd ff:ff:ff:ff:ff:ff
	    inet 10.200.4.3/24 scope global eth0                                                                                                                                                                           
	       valid_lft forever preferred_lft forever                                                                                                                                                                     
	    inet6 fd33:514a:426a:4::3/64 scope global     
	       valid_lft forever preferred_lft forever
	    inet6 fe80::dcc4:57ff:feeb:9b87/64 scope link                                                                                                                                                                  
	       valid_lft forever preferred_lft forever                                                                                                                                                                     
	3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
	    link/ether 5e:bf:f7:5a:7a:7b brd ff:ff:ff:ff:ff:ff
	    inet 185.69.167.139/24 scope global eth1                                                                                                                                                                       
	       valid_lft forever preferred_lft forever                                                                                                                                                                     
	    inet6 2a02:1802:5e:0:5cbf:f7ff:fe5a:7a7b/64 scope global dynamic mngtmpaddr 
	       valid_lft 2592000sec preferred_lft 604800sec
	    inet6 fe80::5cbf:f7ff:fe5a:7a7b/64 scope link                                                                                                                                                                  
	       valid_lft forever preferred_lft forever                                                                                                                                                                     
	4: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
	    link/void                                 
	5: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000                                                                                                                                      
	    link/ipip 0.0.0.0 brd 0.0.0.0                                                                                                                                                                                  
	6: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default qlen 1000
	    link/gre 0.0.0.0 brd 0.0.0.0              
	7: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000
	    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
	8: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
	    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
	9: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
	    link/ipip 0.0.0.0 brd 0.0.0.0
	10: ip6_vti0@NONE: <NOARP> mtu 1428 qdisc noop state DOWN group default qlen 1000
	    link/tunnel6 :: brd :: permaddr 866e:93b2:73ba::
	11: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
	    link/sit 0.0.0.0 brd 0.0.0.0
	12: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default qlen 1000
	    link/tunnel6 :: brd :: permaddr 8eb5:3246:3602::
	13: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default qlen 1000
	    link/gre6 :: brd :: permaddr 7e35:fa9b:c176::
	14: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default 
	    link/ether e2:15:c2:0c:68:c6 brd ff:ff:ff:ff:ff:ff
	    inet 10.42.0.0/32 scope global flannel.1
	       valid_lft forever preferred_lft forever
	    inet6 fe80::e015:c2ff:fe0c:68c6/64 scope link 
	       valid_lft forever preferred_lft forever
	15: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
	    link/ether 5e:3c:4d:e0:99:b9 brd ff:ff:ff:ff:ff:ff
	    inet 10.42.0.1/24 brd 10.42.0.255 scope global cni0
	       valid_lft forever preferred_lft forever
	    inet6 fe80::5c3c:4dff:fee0:99b9/64 scope link 
	       valid_lft forever preferred_lft forever
	16: veth6b447593@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether 06:a2:44:d3:12:0c brd ff:ff:ff:ff:ff:ff link-netns cni-f4ca4468-df27-6a01-2f99-f848485cb01f
	    inet6 fe80::4a2:44ff:fed3:120c/64 scope link 
	       valid_lft forever preferred_lft forever
	18: veth1f4cb751@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether e6:af:b1:56:c4:20 brd ff:ff:ff:ff:ff:ff link-netns cni-fa1fa434-ab04-5ab8-b05d-bae36bef8d54
	    inet6 fe80::e4af:b1ff:fe56:c420/64 scope link 
	       valid_lft forever preferred_lft forever
	19: vethc1929567@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether 52:3d:7d:83:5e:10 brd ff:ff:ff:ff:ff:ff link-netns cni-3bfbe241-cdb2-27f3-e003-b3a74741ac39
	    inet6 fe80::503d:7dff:fe83:5e10/64 scope link 
	       valid_lft forever preferred_lft forever
	20: veth9fb22c46@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether f6:0b:aa:9f:2b:01 brd ff:ff:ff:ff:ff:ff link-netns cni-aa13c79b-6e08-9a6f-8cac-257297cb800e
	    inet6 fe80::f40b:aaff:fe9f:2b01/64 scope link 
	       valid_lft forever preferred_lft forever
	23: veth4e383503@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether 2e:b1:c9:c9:b8:f8 brd ff:ff:ff:ff:ff:ff link-netns cni-efe1de7b-162f-217f-f01e-28ec045eec1e
	    inet6 fe80::2cb1:c9ff:fec9:b8f8/64 scope link 
	       valid_lft forever preferred_lft forever
	24: vethe9638f77@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
	    link/ether ca:63:8e:93:f4:b9 brd ff:ff:ff:ff:ff:ff link-netns cni-46ddf5ad-c91f-c4b5-10ce-0f68e636eec0
	    inet6 fe80::c863:8eff:fe93:f4b9/64 scope link 
	       valid_lft forever preferred_lft forever

And this is ip -6 r

k3os-8627 [~]$ ip -6 r
2a02:1802:5e::/64 dev eth1 proto kernel metric 256 expires 2591999sec pref medium
fd33:514a:426a:4::/64 dev eth0 proto kernel metric 256 pref medium
fc00::/7 via fd33:514a:426a:4::1 dev eth0 metric 1024 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev flannel.1 proto kernel metric 256 pref medium
fe80::/64 dev cni0 proto kernel metric 256 pref medium
fe80::/64 dev veth6b447593 proto kernel metric 256 pref medium
fe80::/64 dev veth1f4cb751 proto kernel metric 256 pref medium
fe80::/64 dev vethc1929567 proto kernel metric 256 pref medium
fe80::/64 dev veth9fb22c46 proto kernel metric 256 pref medium
fe80::/64 dev veth4e383503 proto kernel metric 256 pref medium
fe80::/64 dev vethe9638f77 proto kernel metric 256 pref medium
default via fe80::2e0:ecff:fe7b:7a67 dev eth1 proto ra metric 1024 expires 8sec hoplimit 64 pref high

@delandtj
Copy link
Contributor

strange... I can ping all these IPs, the more , they're in the same segment/prefix, so I can't figure out what's happening to you unless I have access to the vms themselves

@LeeSmet
Copy link
Contributor

LeeSmet commented Feb 24, 2021 •
edited
Loading

deploying a veth pair on the node which imitates the VM network setup works fine to ping the db, so as @delandtj said we'd need access to the vm itself to figure out what is happening. SSH keys are on our user profiles, please add them

@LeeSmet
Copy link
Contributor

LeeSmet commented Feb 24, 2021 •
edited
Loading

Seems at some point the firwalling rules got messed up and use an yggdrasil IP instead of the predicted SLAAC IP. this causes all ip6 from public IPs to be dropped

ip6 saddr . ether saddr != { 300:596a:6375:dea:3c1f:efff:fe86:7d4c . 3e:1f:ef:86:7d:4c } counter packets 10475 bytes 781820 drop

the 300:: ip is an yggdrasil ip which is currently not set, and should be something like 2a02:1802:5e:0:d847:a6ff:fe5b:b303

@LeeSmet LeeSmet changed the title ZDB not reachable from a VDC node and reachable from another on devnet Public IP firewalls wrong IPv6 Feb 24, 2021
LeeSmet added a commit that referenced this issue Feb 25, 2021
@LeeSmet
Fix #1191: Ignore yggdrasil IP as public subnets
cefb3c6 
Signed-off-by: Lee Smet <lee.smet@hotmail.com>
@LeeSmet LeeSmet reopened this Feb 25, 2021
@LeeSmet
Copy link
Contributor

LeeSmet commented Feb 25, 2021 •
edited
Loading

Supposed fix is merged, newly deployed vm's with public IP's should work now (once the update is propagated in about 15 mins)

@DylanVerstraete
Copy link
Contributor

Closing because issue is not happening anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@delandtj delandtj

@LeeSmet LeeSmet

Labels
type_bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@delandtj @OmarElawady @DylanVerstraete @LeeSmet