Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Directed broadcast test fails , packets are dropped at egress #3224

Closed
ArunSaravananBalachandran opened this issue Jul 26, 2019 · 3 comments
Closed

Directed broadcast test fails , packets are dropped at egress #3224

ArunSaravananBalachandran opened this issue Jul 26, 2019 · 3 comments
Assignees
@lguohan

Comments

@ArunSaravananBalachandran
Copy link
Contributor

Description

Directed broadcast test fails for Trident 3 platforms. The packets are dropped at egress.

Steps to reproduce the issue:

  1. Run https://github.com/Azure/sonic-mgmt/blob/master/ansible/roles/test/tasks/dir_bcast.yml

Upon sending the traffic through script , the packet drops are seen at egress

Describe the results you received:

Script logs:

TASK [test : PTF Test - dir_bcast test] ****************************************
task path: /var/johnar/sonic-mgmt/ansible/roles/test/tasks/ptf_runner.yml:42
Friday 26 July 2019  16:24:25 +0000 (0:00:00.131)       0:00:18.018 *********** 
ok: [sonic-s5232-64] => {
    "msg": "ptf --test-dir ptftests dir_bcast_test.BcastTest   --platform-dir ptftests  --platform remote  -t \"testbed_type='t0';router_mac='3c:2c:30:6d:80:00';vlan_info='/root/vlan_info.txt'\"  --relax --debug info --log-file /tmp/dir_bcast.BcastTest.2019-07-26-16:24:25.log 2>&1"
}

TASK [test : command] **********************************************************
task path: /var/johnar/sonic-mgmt/ansible/roles/test/tasks/ptf_runner.yml:45
Friday 26 July 2019  16:24:26 +0000 (0:00:00.181)       0:00:18.199 *********** 
<10.16.210.140> ESTABLISH SSH CONNECTION FOR USER: root
<10.16.210.140> SSH: EXEC sshpass -d15 ssh -C -q -o ControlMaster=auto -o ControlPersist=120s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r 10.16.210.140 'LANG=C LC_ALL=C LC_MESSAGES=C /usr/bin/python'
changed: [sonic-s5232-64 -> 10.16.210.140] => {"changed": true, "cmd": "ptf --test-dir ptftests dir_bcast_test.BcastTest   --platform-dir ptftests  --platform remote  -t \"testbed_type='t0';router_mac='3c:2c:30:6d:80:00';vlan_info='/root/vlan_info.txt'\"  --relax --debug info --log-file /tmp/dir_bcast.BcastTest.2019-07-26-16:24:26.log 2>&1", "delta": "0:00:02.951175", "end": "2019-07-26 16:24:29.218739", "failed": false, "failed_when_result": false, "invocation": {"module_args": {"_raw_params": "ptf --test-dir ptftests dir_bcast_test.BcastTest   --platform-dir ptftests  --platform remote  -t \"testbed_type='t0';router_mac='3c:2c:30:6d:80:00';vlan_info='/root/vlan_info.txt'\"  --relax --debug info --log-file /tmp/dir_bcast.BcastTest.2019-07-26-16:24:26.log 2>&1", "_uses_shell": true, "chdir": "/root", "creates": null, "executable": null, "removes": null, "warn": true}, "module_name": "command"}, "rc": 1, "start": "2019-07-26 16:24:26.267564", "stderr": "", "stdout": "WARNING: No route found for IPv6 destination :: (no default route?)\ndir_bcast_test.BcastTest ... FAIL\n\n======================================================================\nFAIL: dir_bcast_test.BcastTest\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"ptftests/dir_bcast_test.py\", line 177, in runTest\n    self.check_all_dir_bcast()\n  File \"ptftests/dir_bcast_test.py\", line 86, in check_all_dir_bcast\n    self.check_ip_dir_bcast(bcast_ip, dst_port_list)\n  File \"ptftests/dir_bcast_test.py\", line 123, in check_ip_dir_bcast\n    assert (pkt_count == len(dst_port_list))\nAssertionError\n\n----------------------------------------------------------------------\nRan 1 test in 1.926s\n\nFAILED (failures=1)", "stdout_lines": ["WARNING: No route found for IPv6 destination :: (no default route?)", "dir_bcast_test.BcastTest ... FAIL", "", "======================================================================", "FAIL: dir_bcast_test.BcastTest", "----------------------------------------------------------------------", "Traceback (most recent call last):", "  File \"ptftests/dir_bcast_test.py\", line 177, in runTest", "    self.check_all_dir_bcast()", "  File \"ptftests/dir_bcast_test.py\", line 86, in check_all_dir_bcast", "    self.check_ip_dir_bcast(bcast_ip, dst_port_list)", "  File \"ptftests/dir_bcast_test.py\", line 123, in check_ip_dir_bcast", "    assert (pkt_count == len(dst_port_list))", "AssertionError", "", "----------------------------------------------------------------------", "Ran 1 test in 1.926s", "", "FAILED (failures=1)"], "warnings": []}

TASK [test : debug] ************************************************************
task path: /var/johnar/sonic-mgmt/ansible/roles/test/tasks/ptf_runner.yml:52
Friday 26 July 2019  16:24:29 +0000 (0:00:03.221)       0:00:21.421 *********** 
ok: [sonic-s5232-64] => {
    "out.stdout_lines": [
        "WARNING: No route found for IPv6 destination :: (no default route?)", 
        "dir_bcast_test.BcastTest ... FAIL", 
        "", 
        "======================================================================", 
        "FAIL: dir_bcast_test.BcastTest", 
        "----------------------------------------------------------------------", 
        "Traceback (most recent call last):", 
        "  File \"ptftests/dir_bcast_test.py\", line 177, in runTest", 
        "    self.check_all_dir_bcast()", 
        "  File \"ptftests/dir_bcast_test.py\", line 86, in check_all_dir_bcast", 
        "    self.check_ip_dir_bcast(bcast_ip, dst_port_list)", 
        "  File \"ptftests/dir_bcast_test.py\", line 123, in check_ip_dir_bcast", 
        "    assert (pkt_count == len(dst_port_list))", 
        "AssertionError", 
        "", 
        "----------------------------------------------------------------------", 
        "Ran 1 test in 1.926s", 
        "", 
        "FAILED (failures=1)"
    ]
}

DUT logs:

DUT's counters showing TX drops after running the script

root@sonic-s5232-64:~# show interfaces counters
      IFACE    STATE    RX_OK    RX_BPS    RX_UTIL    RX_ERR    RX_DRP    RX_OVR    TX_OK    TX_BPS    TX_UTIL    TX_ERR    TX_DRP    TX_OVR
-----------  -------  -------  --------  ---------  --------  --------  --------  -------  --------  ---------  --------  --------  --------
  Ethernet0        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
  Ethernet4        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
  Ethernet8        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet12        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet16        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet20        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet24        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet28        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet32        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet36        U       72  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet40        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet44        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet48        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet52        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet56        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet60        U       72  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet64        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet68        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet72        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet76        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet80        U       72  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet84        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0         1         0
 Ethernet88        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet92        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
 Ethernet96        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0         1         0
Ethernet100        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet104        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet108        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet112        U      455  0.45 B/s      0.00%         0       246         0      381  0.40 B/s      0.00%         0         0         0
Ethernet116        U      382  0.52 B/s      0.00%         0       246         0      343  0.52 B/s      0.00%         0         0         0
Ethernet120        U      383  0.52 B/s      0.00%         0       247         0      344  0.52 B/s      0.00%         0         0         0
Ethernet124        U      327  0.31 B/s      0.00%         0       247         0      254  0.25 B/s      0.00%         0         0         0
Ethernet128        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet129        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
root@sonic-s5232-64:~#

After sending 400 packets through the script , following are the values of the DUT's counters

root@sonic-s5232-64:~# show interfaces counters
      IFACE    STATE    RX_OK    RX_BPS    RX_UTIL    RX_ERR    RX_DRP    RX_OVR    TX_OK    TX_BPS    TX_UTIL    TX_ERR    TX_DRP    TX_OVR
-----------  -------  -------  --------  ---------  --------  --------  --------  -------  --------  ---------  --------  --------  --------
  Ethernet0        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
  Ethernet4        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
  Ethernet8        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet12        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet16        U       72  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet20        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet24        U       72  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet28        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet32        U       72  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet36        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet40        U      472  0.47 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet44        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet48        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet52        U       72  0.07 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet56        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet60        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet64        U       72  0.07 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet68        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet72        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet76        U       71  0.06 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet80        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet84        U       71  0.06 B/s      0.00%         0        67         0        6  0.01 B/s      0.00%         0       400         0
 Ethernet88        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet92        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
 Ethernet96        U       72  0.07 B/s      0.00%         0        67         0        7  0.01 B/s      0.00%         0       400         0
Ethernet100        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet104        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet108        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet112        U      453  0.45 B/s      0.00%         0       244         0      380  0.40 B/s      0.00%         0         0         0
Ethernet116        U      387  0.53 B/s      0.00%         0       244         0      362  0.56 B/s      0.00%         0         0         0
Ethernet120        U      390  0.54 B/s      0.00%         0       247         0      365  0.56 B/s      0.00%         0         0         0
Ethernet124        U      326  0.31 B/s      0.00%         0       247         0      255  0.26 B/s      0.00%         0         0         0
Ethernet128        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
Ethernet129        X        0  0.00 B/s      0.00%         0         0         0        0  0.00 B/s      0.00%         0         0         0
root@sonic-s5232-64:~#

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):
@prsunny
Copy link
Contributor

prsunny commented Jul 30, 2019

Trident3 enables directed broadcast via fp rules. Can you first check if there is an entry with the action RedirectIpmc

bcmcmd 'fp show' | grep 'Redirect'

@ArunSaravananBalachandran
Copy link
Contributor Author

Trident3 enables directed broadcast via fp rules. Can you first check if there is an entry with the action RedirectIpmc

bcmcmd 'fp show' | grep 'Redirect'

A FP entry with 'RedirectIpmc' action is present and have included the same below.

root@sonic-s5232-64:~# 
root@sonic-s5232-64:~# bcmcmd 'fp show' | grep 'Redirect'
         aset={CosQNew, CosQCpuNew, UcastCosQNew, McastCosQNew, PrioPktCopy, PrioPktNew, PrioPktTos, PrioPktCancel, PrioIntCopy, PrioIntNew, PrioIntTos, PrioIntCancel, DscpNew, DscpCancel, EcnNew, CopyToCpu, CopyToCpuCancel, SwitchToCpuCancel, SwitchToCpuReinstate, TimeStampToCpu, TimeStampToCpuCancel, RedirectPort, RedirectTrunk, RedirectCancel, RedirectPbmp, RedirectIpmc, RedirectMcast, RedirectVlan, RedirectBcastPbmp, EgressMask, EgressPortsAdd, Drop, DropCancel, MirrorOverride, MirrorIngress, MirrorEgress, L3Switch, L3SwitchCancel, DropPrecedence, RpDrop, RpDropCancel, RpDropPrecedence, RpCopyToCpu, RpCopyToCpuCancel, RpSwitchToCpuCancel, RpSwitchToCpuReinstate, RpTimeStampToCpu, RpTimeStampToCpuCancel, RpDscpNew, RpDscpCancel, RpEcnNew, RpPrioPktCopy, RpPrioPktNew, RpPrioPktTos, RpPrioPktCancel, RpCosQNew, RpUcastCosQNew, RpMcastCosQNew, RpPrioIntCopy, RpPrioIntNew, RpPrioIntTos, RpPrioIntCancel, YpDrop, YpDropCancel, YpDropPrecedence, YpCopyToCpu, YpCopyToCpuCancel, YpSwitchToCpuCancel, YpSwitchToCpuReinstate, YpTimeStampToCpu, YpTimeStampToCpuCancel, YpDscpNew, YpDscpCancel, YpEcnNew, YpPrioPktCopy, YpPrioPktNew, YpPrioPktTos, YpPrioPktCancel, YpCosQNew, YpUcastCosQNew, YpMcastCosQNew, YpPrioIntCopy, YpPrioIntNew, YpPrioIntTos, YpPrioIntCancel, ColorIndependent, GpDrop, GpDropCancel, GpDropPrecedence, GpCopyToCpu, GpCopyToCpuCancel, GpSwitchToCpuCancel, GpSwitchToCpuReinstate, GpTimeStampToCpu, GpTimeStampToCpuCancel, GpDscpNew, GpDscpCancel, GpEcnNew, GpTosPrecedenceNew, GpTosPrecedenceCopy, GpPrioPktCopy, GpPrioPktNew, GpPrioPktTos, GpPrioPktCancel, GpCosQNew, GpUcastCosQNew, GpMcastCosQNew, GpPrioIntCopy, GpPrioIntNew, GpPrioIntTos, GpPrioIntCancel, DoNotChangeTtl, DoNotCheckUrpf, NewClassId, MultipathHash, FabricQueue, DynamicHgTrunkCancel, ServicePoolIdNew, RedirectEgrNextHop, EcmpLoadBalanceCancel, EgressClassSelect, HiGigClassSelect, PolicerGroup, StatGroup, NatCancel, Nat, NatEgressOverride, FibreChanVsanId, FibreChanZoneCheckActionCancel, FibreChanIntVsanPri, IngSampleEnable, UnmodifiedPacketRedirectPort, RpIntCongestionNotificationNew, YpIntCongestionNotificationNew, GpIntCongestionNotificationNew, RecoverableDropCancel, FabricEHAddOrUpdate, Dot1pPreserve, GpDot1pPreserve, YpDot1pPreserve, RpDot1pPreserve, DscpPreserve, GpDscpPreserve, YpDscpPreserve, RpDscpPreserve, ChangeL2FieldsCancel, BFDSessionIdNew, CosMapNew, GpCosMapNew, YpCosMapNew, RpCosMapNew, ChangeL2Fields, HgTrunkRandomRoundRobinHashCancel, TrunkRandomRoundRobinHashCancel, EcmpRandomRoundRobinHashCancel, PacketTraceEnable, UntaggedPacketPriorityNew, DoNotCutThrough, ServicePoolIdPrecedenceNew, EgressTimeStampInsert, IngressTimeStampInsert, IngressTimeStampInsertCancel, EgressTimeStampInsertCancel, DynamicEcmpEnable, Dgm, DgmThreshold, DgmBias, DgmCost, DynamicHgTrunkEnable, TrunkResilientHashEnable, TrunkResilientHashDisable, EcmpResilientHashEnable, EcmpResilientHashDisable, HiGigTrunkResilientHashEnable, HiGigTrunkResilientHashDisable, IntEncapEnable, IntTurnAround, AddIngOuterVlanToEgrOuterVlan, Erspan3HdrVlanCosPktCopy, IntEncapUpdate, LoopbackType, NshEncapEnable, NshServicePathId, NshServiceIndex, AssignOpaqueObject1, AssignOpaqueObject2, AssignOpaqueObject3, AssignOpaqueObject4, AssignExtractionCtrlId, AssignEditCtrlId, LoopbackSubtype, EgressFlowControlEnable, EgressFlowEncapEnable},
         aset={CosQNew, CosQCpuNew, UcastCosQNew, McastCosQNew, PrioPktCopy, PrioPktNew, PrioPktTos, PrioPktCancel, PrioIntCopy, PrioIntNew, PrioIntTos, PrioIntCancel, DscpNew, DscpCancel, EcnNew, CopyToCpu, CopyToCpuCancel, SwitchToCpuCancel, SwitchToCpuReinstate, TimeStampToCpu, TimeStampToCpuCancel, RedirectPort, RedirectTrunk, RedirectCancel, RedirectPbmp, RedirectIpmc, RedirectMcast, RedirectVlan, RedirectBcastPbmp, EgressMask, EgressPortsAdd, Drop, DropCancel, MirrorOverride, MirrorIngress, MirrorEgress, L3Switch, L3SwitchCancel, DropPrecedence, RpDrop, RpDropCancel, RpDropPrecedence, RpCopyToCpu, RpCopyToCpuCancel, RpSwitchToCpuCancel, RpSwitchToCpuReinstate, RpTimeStampToCpu, RpTimeStampToCpuCancel, RpDscpNew, RpDscpCancel, RpEcnNew, RpPrioPktCopy, RpPrioPktNew, RpPrioPktTos, RpPrioPktCancel, RpCosQNew, RpUcastCosQNew, RpMcastCosQNew, RpPrioIntCopy, RpPrioIntNew, RpPrioIntTos, RpPrioIntCancel, YpDrop, YpDropCancel, YpDropPrecedence, YpCopyToCpu, YpCopyToCpuCancel, YpSwitchToCpuCancel, YpSwitchToCpuReinstate, YpTimeStampToCpu, YpTimeStampToCpuCancel, YpDscpNew, YpDscpCancel, YpEcnNew, YpPrioPktCopy, YpPrioPktNew, YpPrioPktTos, YpPrioPktCancel, YpCosQNew, YpUcastCosQNew, YpMcastCosQNew, YpPrioIntCopy, YpPrioIntNew, YpPrioIntTos, YpPrioIntCancel, ColorIndependent, GpDrop, GpDropCancel, GpDropPrecedence, GpCopyToCpu, GpCopyToCpuCancel, GpSwitchToCpuCancel, GpSwitchToCpuReinstate, GpTimeStampToCpu, GpTimeStampToCpuCancel, GpDscpNew, GpDscpCancel, GpEcnNew, GpTosPrecedenceNew, GpTosPrecedenceCopy, GpPrioPktCopy, GpPrioPktNew, GpPrioPktTos, GpPrioPktCancel, GpCosQNew, GpUcastCosQNew, GpMcastCosQNew, GpPrioIntCopy, GpPrioIntNew, GpPrioIntTos, GpPrioIntCancel, DoNotChangeTtl, DoNotCheckUrpf, NewClassId, MultipathHash, FabricQueue, DynamicHgTrunkCancel, ServicePoolIdNew, RedirectEgrNextHop, EcmpLoadBalanceCancel, EgressClassSelect, HiGigClassSelect, PolicerGroup, StatGroup, NatCancel, Nat, NatEgressOverride, FibreChanVsanId, FibreChanZoneCheckActionCancel, FibreChanIntVsanPri, IngSampleEnable, UnmodifiedPacketRedirectPort, RpIntCongestionNotificationNew, YpIntCongestionNotificationNew, GpIntCongestionNotificationNew, RecoverableDropCancel, FabricEHAddOrUpdate, Dot1pPreserve, GpDot1pPreserve, YpDot1pPreserve, RpDot1pPreserve, DscpPreserve, GpDscpPreserve, YpDscpPreserve, RpDscpPreserve, ChangeL2FieldsCancel, BFDSessionIdNew, CosMapNew, GpCosMapNew, YpCosMapNew, RpCosMapNew, ChangeL2Fields, HgTrunkRandomRoundRobinHashCancel, TrunkRandomRoundRobinHashCancel, EcmpRandomRoundRobinHashCancel, PacketTraceEnable, UntaggedPacketPriorityNew, DoNotCutThrough, ServicePoolIdPrecedenceNew, EgressTimeStampInsert, IngressTimeStampInsert, IngressTimeStampInsertCancel, EgressTimeStampInsertCancel, DynamicEcmpEnable, Dgm, DgmThreshold, DgmBias, DgmCost, DynamicHgTrunkEnable, TrunkResilientHashEnable, TrunkResilientHashDisable, EcmpResilientHashEnable, EcmpResilientHashDisable, HiGigTrunkResilientHashEnable, HiGigTrunkResilientHashDisable, IntEncapEnable, IntTurnAround, AddIngOuterVlanToEgrOuterVlan, Erspan3HdrVlanCosPktCopy, IntEncapUpdate, LoopbackType, NshEncapEnable, NshServicePathId, NshServiceIndex, AssignOpaqueObject1, AssignOpaqueObject2, AssignOpaqueObject3, AssignOpaqueObject4, AssignExtractionCtrlId, AssignEditCtrlId, LoopbackSubtype, EgressFlowControlEnable, EgressFlowEncapEnable},
         aset={CosQNew, CosQCpuNew, UcastCosQNew, McastCosQNew, PrioPktCopy, PrioPktNew, PrioPktTos, PrioPktCancel, PrioIntCopy, PrioIntNew, PrioIntTos, PrioIntCancel, DscpNew, DscpCancel, EcnNew, CopyToCpu, CopyToCpuCancel, SwitchToCpuCancel, SwitchToCpuReinstate, TimeStampToCpu, TimeStampToCpuCancel, RedirectPort, RedirectTrunk, RedirectCancel, RedirectPbmp, RedirectIpmc, RedirectMcast, RedirectVlan, RedirectBcastPbmp, EgressMask, EgressPortsAdd, Drop, DropCancel, MirrorOverride, MirrorIngress, MirrorEgress, L3Switch, L3SwitchCancel, DropPrecedence, RpDrop, RpDropCancel, RpDropPrecedence, RpCopyToCpu, RpCopyToCpuCancel, RpSwitchToCpuCancel, RpSwitchToCpuReinstate, RpTimeStampToCpu, RpTimeStampToCpuCancel, RpDscpNew, RpDscpCancel, RpEcnNew, RpPrioPktCopy, RpPrioPktNew, RpPrioPktTos, RpPrioPktCancel, RpCosQNew, RpUcastCosQNew, RpMcastCosQNew, RpPrioIntCopy, RpPrioIntNew, RpPrioIntTos, RpPrioIntCancel, YpDrop, YpDropCancel, YpDropPrecedence, YpCopyToCpu, YpCopyToCpuCancel, YpSwitchToCpuCancel, YpSwitchToCpuReinstate, YpTimeStampToCpu, YpTimeStampToCpuCancel, YpDscpNew, YpDscpCancel, YpEcnNew, YpPrioPktCopy, YpPrioPktNew, YpPrioPktTos, YpPrioPktCancel, YpCosQNew, YpUcastCosQNew, YpMcastCosQNew, YpPrioIntCopy, YpPrioIntNew, YpPrioIntTos, YpPrioIntCancel, ColorIndependent, GpDrop, GpDropCancel, GpDropPrecedence, GpCopyToCpu, GpCopyToCpuCancel, GpSwitchToCpuCancel, GpSwitchToCpuReinstate, GpTimeStampToCpu, GpTimeStampToCpuCancel, GpDscpNew, GpDscpCancel, GpEcnNew, GpTosPrecedenceNew, GpTosPrecedenceCopy, GpPrioPktCopy, GpPrioPktNew, GpPrioPktTos, GpPrioPktCancel, GpCosQNew, GpUcastCosQNew, GpMcastCosQNew, GpPrioIntCopy, GpPrioIntNew, GpPrioIntTos, GpPrioIntCancel, DoNotChangeTtl, DoNotCheckUrpf, NewClassId, MultipathHash, FabricQueue, DynamicHgTrunkCancel, ServicePoolIdNew, RedirectEgrNextHop, EcmpLoadBalanceCancel, EgressClassSelect, HiGigClassSelect, PolicerGroup, StatGroup, NatCancel, Nat, NatEgressOverride, FibreChanVsanId, FibreChanZoneCheckActionCancel, FibreChanIntVsanPri, IngSampleEnable, UnmodifiedPacketRedirectPort, RpIntCongestionNotificationNew, YpIntCongestionNotificationNew, GpIntCongestionNotificationNew, RecoverableDropCancel, FabricEHAddOrUpdate, Dot1pPreserve, GpDot1pPreserve, YpDot1pPreserve, RpDot1pPreserve, DscpPreserve, GpDscpPreserve, YpDscpPreserve, RpDscpPreserve, ChangeL2FieldsCancel, BFDSessionIdNew, CosMapNew, GpCosMapNew, YpCosMapNew, RpCosMapNew, ChangeL2Fields, HgTrunkRandomRoundRobinHashCancel, TrunkRandomRoundRobinHashCancel, EcmpRandomRoundRobinHashCancel, PacketTraceEnable, UntaggedPacketPriorityNew, DoNotCutThrough, ServicePoolIdPrecedenceNew, EgressTimeStampInsert, IngressTimeStampInsert, IngressTimeStampInsertCancel, EgressTimeStampInsertCancel, DynamicEcmpEnable, Dgm, DgmThreshold, DgmBias, DgmCost, DynamicHgTrunkEnable, TrunkResilientHashEnable, TrunkResilientHashDisable, EcmpResilientHashEnable, EcmpResilientHashDisable, HiGigTrunkResilientHashEnable, HiGigTrunkResilientHashDisable, IntEncapEnable, IntTurnAround, AddIngOuterVlanToEgrOuterVlan, Erspan3HdrVlanCosPktCopy, IntEncapUpdate, LoopbackType, NshEncapEnable, NshServicePathId, NshServiceIndex, AssignOpaqueObject1, AssignOpaqueObject2, AssignOpaqueObject3, AssignOpaqueObject4, AssignExtractionCtrlId, AssignEditCtrlId, LoopbackSubtype, EgressFlowControlEnable, EgressFlowEncapEnable},
         aset={CosQNew, CosQCpuNew, UcastCosQNew, McastCosQNew, PrioPktCopy, PrioPktNew, PrioPktTos, PrioPktCancel, PrioIntCopy, PrioIntNew, PrioIntTos, PrioIntCancel, DscpNew, DscpCancel, EcnNew, CopyToCpu, CopyToCpuCancel, SwitchToCpuCancel, SwitchToCpuReinstate, TimeStampToCpu, TimeStampToCpuCancel, RedirectPort, RedirectTrunk, RedirectCancel, RedirectPbmp, RedirectIpmc, RedirectMcast, RedirectVlan, RedirectBcastPbmp, EgressMask, EgressPortsAdd, Drop, DropCancel, MirrorOverride, MirrorIngress, MirrorEgress, L3Switch, L3SwitchCancel, DropPrecedence, RpDrop, RpDropCancel, RpDropPrecedence, RpCopyToCpu, RpCopyToCpuCancel, RpSwitchToCpuCancel, RpSwitchToCpuReinstate, RpTimeStampToCpu, RpTimeStampToCpuCancel, RpDscpNew, RpDscpCancel, RpEcnNew, RpPrioPktCopy, RpPrioPktNew, RpPrioPktTos, RpPrioPktCancel, RpCosQNew, RpUcastCosQNew, RpMcastCosQNew, RpPrioIntCopy, RpPrioIntNew, RpPrioIntTos, RpPrioIntCancel, YpDrop, YpDropCancel, YpDropPrecedence, YpCopyToCpu, YpCopyToCpuCancel, YpSwitchToCpuCancel, YpSwitchToCpuReinstate, YpTimeStampToCpu, YpTimeStampToCpuCancel, YpDscpNew, YpDscpCancel, YpEcnNew, YpPrioPktCopy, YpPrioPktNew, YpPrioPktTos, YpPrioPktCancel, YpCosQNew, YpUcastCosQNew, YpMcastCosQNew, YpPrioIntCopy, YpPrioIntNew, YpPrioIntTos, YpPrioIntCancel, ColorIndependent, GpDrop, GpDropCancel, GpDropPrecedence, GpCopyToCpu, GpCopyToCpuCancel, GpSwitchToCpuCancel, GpSwitchToCpuReinstate, GpTimeStampToCpu, GpTimeStampToCpuCancel, GpDscpNew, GpDscpCancel, GpEcnNew, GpTosPrecedenceNew, GpTosPrecedenceCopy, GpPrioPktCopy, GpPrioPktNew, GpPrioPktTos, GpPrioPktCancel, GpCosQNew, GpUcastCosQNew, GpMcastCosQNew, GpPrioIntCopy, GpPrioIntNew, GpPrioIntTos, GpPrioIntCancel, DoNotChangeTtl, DoNotCheckUrpf, NewClassId, MultipathHash, FabricQueue, DynamicHgTrunkCancel, ServicePoolIdNew, RedirectEgrNextHop, EcmpLoadBalanceCancel, EgressClassSelect, HiGigClassSelect, PolicerGroup, StatGroup, NatCancel, Nat, NatEgressOverride, FibreChanVsanId, FibreChanZoneCheckActionCancel, FibreChanIntVsanPri, IngSampleEnable, UnmodifiedPacketRedirectPort, RpIntCongestionNotificationNew, YpIntCongestionNotificationNew, GpIntCongestionNotificationNew, RecoverableDropCancel, FabricEHAddOrUpdate, Dot1pPreserve, GpDot1pPreserve, YpDot1pPreserve, RpDot1pPreserve, DscpPreserve, GpDscpPreserve, YpDscpPreserve, RpDscpPreserve, ChangeL2FieldsCancel, BFDSessionIdNew, CosMapNew, GpCosMapNew, YpCosMapNew, RpCosMapNew, ChangeL2Fields, HgTrunkRandomRoundRobinHashCancel, TrunkRandomRoundRobinHashCancel, EcmpRandomRoundRobinHashCancel, PacketTraceEnable, UntaggedPacketPriorityNew, DoNotCutThrough, ServicePoolIdPrecedenceNew, EgressTimeStampInsert, IngressTimeStampInsert, IngressTimeStampInsertCancel, EgressTimeStampInsertCancel, DynamicEcmpEnable, Dgm, DgmThreshold, DgmBias, DgmCost, DynamicHgTrunkEnable, TrunkResilientHashEnable, TrunkResilientHashDisable, EcmpResilientHashEnable, EcmpResilientHashDisable, HiGigTrunkResilientHashEnable, HiGigTrunkResilientHashDisable, IntEncapEnable, IntTurnAround, AddIngOuterVlanToEgrOuterVlan, Erspan3HdrVlanCosPktCopy, IntEncapUpdate, LoopbackType, NshEncapEnable, NshServicePathId, NshServiceIndex, AssignOpaqueObject1, AssignOpaqueObject2, AssignOpaqueObject3, AssignOpaqueObject4, AssignExtractionCtrlId, AssignEditCtrlId, LoopbackSubtype, EgressFlowControlEnable, EgressFlowEncapEnable},
         action={act=RedirectIpmc, param0=33554433(0x2000001), param1=0(0), param2=0(0), param3=0(0)}
root@sonic-s5232-64:~# 
root@sonic-s5232-64:~# 
root@sonic-s5232-64:~# 
root@sonic-s5232-64:~# bcmcmd 'fp show entry 0x0000008d '
fp show entry 0x0000008d 
EID 0x0000008d: gid=0x1,
         slice=0, slice_idx=0, part =0, prio=0x64, flags=0x10602, Installed, Enabled, color_indep=1
         slice=1, slice_idx=0, part =1, prio=0x64, flags=0x10604, Installed, Enabled, color_indep=1

 DstMac 
    Part:0 Offset0: 64 Width0:  16 
    Part:0 Offset1: 36 Width1:   4 
    Part:0 Offset2: 40 Width2:   4 
    Part:0 Offset3: 44 Width3:   4 
    Part:0 Offset4: 80 Width4:   4 
    Part:0 Offset5: 116 Width5:   4 
    Part:0 Offset6: 120 Width6:   4 
    Part:0 Offset7: 124 Width7:   4 
    Part:0 Offset8: 128 Width8:   4 
    DATA=0x00003c2c 306d8000 
    MASK=0x0000ffff ffffffff 
 DstIp 
    Part:1 Offset0:  0 Width0:  16 
    Part:1 Offset1: 20 Width1:  16 
    DATA=0xc0a807ff 
    MASK=0xffffffff 
 InPorts 
     DATA=0x0000000000000000000000000000000000000000000000000000000000000000
     MASK=0x0000000000000000000000000000000000000000000000000000000000000000
 _InPorts_0 
   Instance=0
     DATA=0x0000000000
     MASK=0x0000000000
   Instance=1
     DATA=0x0000000000
     MASK=0x0000000000
 _InPorts_1 
   Instance=0
     DATA=0x0
     MASK=0x0
   Instance=1
     DATA=0x0
     MASK=0x0
 _InPorts_2 
   Instance=0
     DATA=0x0
     MASK=0x0
   Instance=1
     DATA=0x0
     MASK=0x0
 L3DestRouteHit 
    Part:0 Offset0: 158 Width0:   1 
    DATA=0x00000001 
    MASK=0x00000001 
         action={act=RedirectIpmc, param0=33554433(0x2000001), param1=0(0), param2=0(0), param3=0(0)}
         policer=
         statistics=NULL
         Extended statistics=NULL

drivshell>
root@sonic-s5232-64:~# 
root@sonic-s5232-64:~#

@justin-lu-lyc
Copy link

Hi , I met the same issue.
The error log is almost the same with the one ArunSaravananBalachandran provided.
Does anyone have any thought for this issue?

mssonicbld added a commit that referenced this issue Mar 22, 2024
@mssonicbld
[submodule] Update submodule sonic-utilities to the latest HEAD autom…
23fd671 
…atically (#18437)

#### Why I did it
src/sonic-utilities
```
* fb4a090d - (HEAD -> 202311, origin/202311) [config] Add Table hard dependency check (#3159) (4 hours ago) [jingwenxie]
* 86f3de58 - [ipintutil]Handle exception in show ip interfaces command (#3182) (4 hours ago) [Sudharsan Dhamal Gopalarathnam]
* 5bfc3b40 - [Techsupport]Handle SAI kv pair if present in sai common profile (#3196) (4 hours ago) [Sudharsan Dhamal Gopalarathnam]
* f4ef7681 - [fast/warm-reboot] Put ERR message in syslog when a failure is seen (#3186) (4 hours ago) [Vaibhav Hemant Dixit]
* db36df24 - [Bug] Fix fw_setenv illegel character issue (#3201) (4 hours ago) [xumia]
* a58b78c8 - [config] Add YANG alerting for override (#3188) (4 hours ago) [jingwenxie]
* 943a6846 - Update port2alias (#3217) (4 hours ago) [abdosi]
* 3236fbf4 - [show] Update show run all to cover all asic config in multiasic (#3148) (#3224) (5 hours ago) [mssonicbld]
```
#### How I did it
#### How to verify it
#### Description for the changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lguohan lguohan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@lguohan @prsunny @justin-lu-lyc @ArunSaravananBalachandran