Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Route-Map: Removing + readding interface address doesn't reinstall routes dependent on the intf address in the kernel #17677

Open
2 tasks done
mikemallin opened this issue Dec 19, 2024 · 0 comments
Open
2 tasks done

Route-Map: Removing + readding interface address doesn't reinstall routes dependent on the intf address in the kernel #17677

mikemallin opened this issue Dec 19, 2024 · 0 comments
Labels
triage Needs further investigation

Comments

@mikemallin
Copy link
Contributor

mikemallin commented Dec 19, 2024 •
edited
Loading

Description

Routes that are programmed in the kernel using a route-map source address are not re-installed after the source address is removed and readded to a loopback interface.

Version

Reproduced on a very recent master branch:

topotests mmallin$ git log -n 1
commit 6c3e1e4fd636e6cdc9a14c4d19de26784a6b890e (HEAD -> master, origin/master, origin/HEAD)
Merge: 76e0c8459 b94e72048
Author: Donald Sharp <donaldsharp72@gmail.com>
Date:   Tue Dec 17 14:44:18 2024 -0500

    Merge pull request #17654 from mjstapp/remove_bgp_pRN
    
    bgpd: remove unneeded printfrr reg for pRN


torm11# show version
FRRouting 10.3-dev-my-manual-build (torm11) on Linux(6.8.0-45-generic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--prefix=/usr' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--enable-vtysh' '--enable-pimd' '--enable-sharpd' '--enable-multipath=64' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-snmp=agentx' '--enable-scripting' '--with-pkg-extra-version=-my-manual-build'

This originally seen on SONiC 202305:

leaf0# show ver
FRRouting 8.5.1 (leaf0) on Linux(5.10.0-23-2-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--disable-rpki' '--disable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

How to reproduce

  1. Checkout the following branch for a topotest that has all the appropriate configuration: git clone https://github.com/mikemallin/frr.git -b dev/bgp-evpn-mh-overlay-peering
    NOTE: This is incomplete and should only be used for launching a topology with configuration
  2. Launch the topotest: sudo -E pytest -s --pause --vtysh=torm11 --shell=torm11 bgp_evpn_mh_overlay_peering
  3. Observe the following routes installed in the kernel on torm11:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
2.2.2.2 nhid 33 proto bgp src 1.1.1.1 metric 20          
        nexthop via 192.168.5.1 dev torm11-eth1 weight 1 
        nexthop via 192.168.1.1 dev torm11-eth0 weight 1 
3.3.3.3 nhid 33 proto bgp src 1.1.1.1 metric 20          
        nexthop via 192.168.5.1 dev torm11-eth1 weight 1 
        nexthop via 192.168.1.1 dev torm11-eth0 weight 1 
4.4.4.4 nhid 33 proto bgp src 1.1.1.1 metric 20          
        nexthop via 192.168.5.1 dev torm11-eth1 weight 1 
        nexthop via 192.168.1.1 dev torm11-eth0 weight 1 
10.10.10.10 nhid 34 via 192.168.1.1 dev torm11-eth0 proto bgp src 1.1.1.1 metric 20  
20.20.20.20 nhid 32 via 192.168.5.1 dev torm11-eth1 proto bgp src 1.1.1.1 metric 20     
<snip>
  1. Observe the same routes in zebra:
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
L * 1.1.1.1/32 is directly connected, lo, weight 1, 00:14:15
C>* 1.1.1.1/32 is directly connected, lo, weight 1, 00:14:15
B>* 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
  *                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
  *                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:02
  *                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
B>* 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:14:03
B>* 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:14:02
  1. Remove the source address 1.1.1.1/32 from interface lo on torm11
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr del 1.1.1.1/32 dev lo
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.100.15/32 brd 192.168.100.15 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
  1. Observe the routes from above in the kernel are no longer there:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
45.0.0.0/24 dev vlan1000 proto kernel scope link src 45.0.0.2 
45.0.0.0/24 dev vlan1000-v0 proto kernel scope link src 45.0.0.1 
192.168.1.0/24 dev torm11-eth0 proto kernel scope link src 192.168.1.2 
192.168.5.0/24 dev torm11-eth1 proto kernel scope link src 192.168.5.2 
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11#
  1. The routes are still in zebra
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
B>r 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:18
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
B>r 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:16:19
B>r 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:16:18
<snip>
  1. Re-add the address 1.1.1.1/32 to interface lo:
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr add 1.1.1.1/32 dev lo
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.100.15/32 brd 192.168.100.15 scope global lo
       valid_lft forever preferred_lft forever
    inet 1.1.1.1/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
  1. The routes are not re-installed in the kernel (after waiting 5m)
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# ip route
45.0.0.0/24 dev vlan1000 proto kernel scope link src 45.0.0.2 
45.0.0.0/24 dev vlan1000-v0 proto kernel scope link src 45.0.0.1 
192.168.1.0/24 dev torm11-eth0 proto kernel scope link src 192.168.1.2 
192.168.5.0/24 dev torm11-eth1 proto kernel scope link src 192.168.5.2 
root@torm11:/tmp/topotests/bgp_evpn_mh_overlay_peering.test_evpn_mh/torm11# 

torm11# show bgp neigh 10.10.10.10
BGP neighbor is 10.10.10.10, remote AS 65101, local AS 65002, external link
  Local Role: undefined
  Remote Role: undefined
Hostname: leaf1
 Member of peer-group OVERLAY for session parameters
  BGP version 4, remote router ID 0.0.0.0, local router ID 192.168.100.15
  BGP state = Active
  Last read 00:05:38, Last write 00:05:29
  Hold time is 10 seconds, keepalive interval is 3 seconds
  Configured hold time is 10 seconds, keepalive interval is 3 seconds
  Configured tcp-mss is 0, synced tcp-mss is 0
  Configured conditional advertisements interval is 60 seconds
  Graceful restart information:
    Local GR Mode: Helper*
    Remote GR Mode: NotApplicable
    R bit: False
    N bit: False
    Timers:
      Configured Restart Time(sec): 120
      Received Restart Time(sec): 120
      Configured LLGR Stale Path Time(sec): 0
  Message statistics:
    Inq depth is 0
    Outq depth is 0
                         Sent       Rcvd
    Opens:                  2          2
    Notifications:          2          0
    Updates:               69         62
    Keepalives:           354        348
    Route Refresh:          0          0
    Capability:             0          0
    Total:                427        412
  Minimum time between advertisement runs is 0 seconds
  Update source is lo

 For address family: L2VPN EVPN
  OVERLAY peer-group member
  Not part of any update group
  NEXT_HOP is propagated unchanged to this neighbor
  Community attribute sent to this neighbor(all)
  advertise-all-vni
  0 accepted prefixes

  Connections established 2; dropped 2
  Last reset 00:05:28,  No path to specified Neighbor (FRRouting/10.3-dev-my-manual-build)
  External BGP neighbor may be up to 255 hops away.
Local host: 1.1.1.1, Local port: 38851
Foreign host: 10.10.10.10, Foreign port: 179
Nexthop: 1.1.1.1
Nexthop global: ::
Nexthop local: ::
BGP connection: non shared network
BGP Connect Retry Timer in Seconds: 10
Next connect timer due in 2 seconds
Read thread: off  Write thread: off  FD used: -1
  1. Zebra still sees the routes as rejected and doesn't re-install them
torm11# show ip route
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF default:
L * 1.1.1.1/32 is directly connected, lo, weight 1, 00:04:11
C>* 1.1.1.1/32 is directly connected, lo, weight 1, 00:04:11
B>r 2.2.2.2/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 3.3.3.3/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 4.4.4.4/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:39
  r                   via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
B>r 10.10.10.10/32 [20/0] via 192.168.1.1, torm11-eth0, rmapsrc 1.1.1.1, weight 1, 00:21:40
B>r 20.20.20.20/32 [20/0] via 192.168.5.1, torm11-eth1, rmapsrc 1.1.1.1, weight 1, 00:21:39
  1. The system can be recovered by issuing clear bgp *

Expected behavior

After step 8, I was expecting the routes to be re-installed in the kernel like in step 3.

Actual behavior

See above. The routes that have the source address set are not re-installed in the kernel.

Additional context

No response

Checklist

  • I have searched the open issues for this bug.
  • I have not included sensitive information in this report.
@mikemallin mikemallin added the triage Needs further investigation label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triage Needs further investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mikemallin