[sflow + dropmon] added the ENABLE_SFLOW_DROPMON build flag. Added patches for sflow repo. #10370

vadymhlushko-mlnx · 2022-03-28T12:58:23Z

DEPENDS ON: sonic-linux-kernel/pull/276, sonic-linux-kernel/pull/281, sonic-linux-kernel/pull/284

Why I did it

Added a capability of monitoring dropped packets for the sFlow daemon in order to improve network - monitoring, diagnostic, and troubleshooting. The drop monitor service allows the sFlow daemon to export another type of sample - dropped packets as Discard samples alongside Counter samples and Packet Flow samples.

How I did it

Added the SONiC build flag - ENABLE_SFLOW_DROPMON which will allow compile the hsflowd daemon with the dropmon kernel module.

How to verify it

Burn an image with the ENABLE_SFLOW_DROPMON=y flag to the switch.
Configure the sFlow on the switch

root@sonic:/home/admin# config feature state sflow enabled
root@sonic:/home/admin# config sflow enable
root@sonic:/home/admin# config sflow collector add collector0 30.1.1.2 --port 6343
root@sonic:/home/admin# config sflow polling-interval 20
root@sonic:/home/admin# config sflow interface disable all
root@sonic:/home/admin# config sflow interface enable Ethernet4
root@sonic:/home/admin# config sflow interface sample-rate Ethernet4 256

Configure the Host (PTF)

root@a8f0f3e20879:/# ifconfig eth1 30.1.1.2/24
root@a8f0f3e20879:/# ifconfig eth2 40.1.1.2/24

Launch the sflowtool on Host (PTF)

sflowtool -J -p 6343 > drop_packets_1k

Send the dropped packets from Host (PTF) to the switch via scapy

root@a8f0f3e20879:/# scapy
Welcome to Scapy (2.2.0-dev)
>>> pkt_nok = Ether(src="00:00:00:00:00", dst="SWITCH_MAC_address")/IP(src="0.0.0.0", dst="40.1.1.5")/TCP()
>>> sendp(pkt_nok, iface="eth2", count=1000)

Check the L3 counters on the switch

root@sonic:/home/admin# show interfaces counters rif
         IFACE    RX_OK      RX_BPS    RX_PPS    RX_ERR    TX_OK    TX_BPS    TX_PPS    TX_ERR
--------------  -------  ----------  --------  --------  -------  --------  --------  --------
     Ethernet2       24  459.32 B/s    0.82/s         1        0  0.00 B/s    0.00/s         0
     Ethernet4        0    0.00 B/s    0.00/s     1,000        0  0.00 B/s    0.00/s         0

Check the samples that were captured by the sflowtool on the Host (PTF)

root@a8f0f3e20879:/# cat drop_packets_1k
...
  {
   "sampleType_tag":"0:5",
   "sampleType":"DISCARD",
   "sampleSequenceNo":"309",
   "sourceId":"0:127",
   "dropEvents":"32",
   "inputPort":"127",
   "outputPort":"0",
   "discardCode":"285",
   "discardReason":"unknown_l3",
   "elements":[{
     "discarded_flowBlock_tag":"0:1038",
     "discarded_extendedType":"function",
     "discarded_symbol":"ip_rcv_finish_core.constprop.0+0x1c4/0x"
    },
    {
     "discarded_flowBlock_tag":"0:1",
     "discarded_flowSampleType":"HEADER",
     "discarded_headerProtocol":"1",
     "discarded_sampledPacketSize":"54",
     "discarded_strippedBytes":"4",
     "discarded_headerLen":"54",
     "discarded_headerBytes":"98-03-9B-94-D4-80-00-00-00-00-00-00-08-00-45-00-00-28-00-01-00-00-40-06-51-CA-00-00-00-00-28-01-01-05-00-14-00-50-00-00-00-00-00-00-00-00-50-02-20-00-66-79-00-00",
     "discarded_dstMAC":"98039b94d480",
     "discarded_srcMAC":"000000000000",
     "discarded_IPSize":"36",
     "discarded_ip.tot_len":"40",
     "discarded_srcIP":"0.0.0.0",
     "discarded_dstIP":"40.1.1.5",
     "discarded_IPProtocol":"6",
     "discarded_IPTOS":"0",
     "discarded_IPTTL":"64",
     "discarded_IPID":"256",
     "discarded_TCPSrcPort":"20",
     "discarded_TCPDstPort":"80",
     "discarded_TCPFlags":"2"
    }
   ]
  }

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

… hsflowd Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

prsunny · 2022-03-28T19:03:20Z

@padmanarayana, @jeff-yin, could someone review?

vadymhlushko-mlnx · 2022-04-04T15:48:45Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2022-04-04T15:48:47Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2022-04-04T15:48:59Z

Azure Pipelines successfully started running 1 pipeline(s).

This reverts commit f4aef7efa6fd65cf6d63960eb312ff662a71fa42.

liat-grozovik · 2022-04-22T08:08:32Z

@padmanarayana, @jeff-yin kindly reminder to review.

vadymhlushko-mlnx · 2022-05-19T11:01:22Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2022-05-19T11:01:23Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2022-05-19T11:01:36Z

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

jeff-yin · 2022-05-26T19:48:56Z

Latest comments (discussed in e-mail):

It’s not clear if the Discard samples are counted against the flow samples (whose rate is specified by “sample-rate” option in the CLI).
Also, the HLD may need an update.

@prsunny to schedule a call with @vadymhlushko-mlnx and @padmanarayana to clarify/discuss.

vadymhlushko-mlnx · 2022-05-27T08:38:51Z

Latest comments (discussed in e-mail):

It’s not clear if the Discard samples are counted against the flow samples (whose rate is specified by “sample-rate” option in the CLI).
Also, the HLD may need an update.

@prsunny to schedule a call with @vadymhlushko-mlnx and @padmanarayana to clarify/discuss.

The sample-rate option could only configure FLOWSAMPLE samples and has nothing to do with DISCARD samples.

For now, there is no CLI for DISCARD samples, the only way to configure something for them is to use the next instruction:

Disable a sFlow
root@sonic:/home/admin# config sflow disable
Go to the sflow docker container
root@sonic:/home/admin# docker exec -it sflow bash
Mannually edit the /etc/hsflowd.conf file
For example, there is a line in /etc/hsflowd.conf

dropmon { group=1 start=on limit=1000 max=10000 hw=on sw=on }

The user could change the

limit [number] - a number that determines how many dropped packets per second could process the dropmon kernel module
max [number] - a number that determines the buffer in the kernel for dropped packets.
hw [on/off] - a flag that enables or disables the interception process of packets that were dropped by the switch ASIC
sw [on/off] - a flag that enables or disables the interception process of packets that were dropped by the Linux kernel stack

Exit the sflow docker and execute sflow enable command in order to apply the changes
root@sonic:/home/admin# config sflow enable

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

vadymhlushko-mlnx · 2022-06-07T07:59:13Z

@padmanarayana, @jeff-yin, kind reminder to review

padmanarayana · 2022-06-08T00:17:42Z

Thanks for the clarifications!

jeff-yin · 2022-06-08T00:21:26Z

Approved -- please seek additional approval from a reviewer with write access to the repo. Thanks!

vadymhlushko-mlnx · 2022-06-08T08:31:22Z

@prsunny could you please merge?

vadymhlushko-mlnx · 2022-06-10T12:30:33Z

@prsunny could you please merge?

kind reminder

…tches for sflow repo. (#10370) * [sflow + dropmon] added INCLUDE_SFLOW_DROPMON flag, added patches for hsflowd *Added a capability of monitoring dropped packets for the sFlow daemon in order to improve network - monitoring, diagnostic, and troubleshooting. The drop monitor service allows the sFlow daemon to export another type of sample - dropped packets as Discard samples alongside Counter samples and Packet Flow samples. Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

…tches for sflow repo. (sonic-net#10370) * [sflow + dropmon] added INCLUDE_SFLOW_DROPMON flag, added patches for hsflowd *Added a capability of monitoring dropped packets for the sFlow daemon in order to improve network - monitoring, diagnostic, and troubleshooting. The drop monitor service allows the sFlow daemon to export another type of sample - dropped packets as Discard samples alongside Counter samples and Packet Flow samples. Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

vadymhlushko-mlnx added 8 commits March 28, 2022 12:34

[sflow + dropmon] added INCLUDE_SFLOW_DROPMON flag, added patches for…

27fe9d3

… hsflowd Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

Fixes include -> enable

2f147b7

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

fixes

6bc243d

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

removed debug trace

da468de

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

review fixes

251e832

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

added --force patch

2c25b0e

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

Fixed export of dropmon flag

55989c1

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

ONLY FOR CI, ENABLED_SFLOW_DROPMON=y

53a55af

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

vadymhlushko-mlnx requested review from qiluo-msft, xumia and lguohan as code owners March 28, 2022 12:58

vadymhlushko-mlnx force-pushed the sflow_dropmon branch 3 times, most recently from 6a4b930 to 769b1b0 Compare April 7, 2022 10:36

Revert "ONLY FOR CI, ENABLED_SFLOW_DROPMON=y"

f6ebf35

This reverts commit f4aef7efa6fd65cf6d63960eb312ff662a71fa42.

vadymhlushko-mlnx force-pushed the sflow_dropmon branch from 769b1b0 to f6ebf35 Compare April 8, 2022 13:36

dprital requested a review from prsunny May 9, 2022 16:54

vadymhlushko-mlnx force-pushed the sflow_dropmon branch 3 times, most recently from 0808bf1 to d062643 Compare May 16, 2022 07:49

Added sw and hw modes in hsflowd.conf

b8fe833

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

vadymhlushko-mlnx force-pushed the sflow_dropmon branch from d062643 to b8fe833 Compare May 20, 2022 15:54

vadymhlushko-mlnx force-pushed the sflow_dropmon branch from 82b0934 to f3d3e7e Compare May 30, 2022 16:00

vadymhlushko-mlnx requested a review from a team as a code owner May 30, 2022 16:00

Enabled HW drops, disabled SW drops

586e108

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>

vadymhlushko-mlnx force-pushed the sflow_dropmon branch from f3d3e7e to 586e108 Compare June 6, 2022 06:58

padmanarayana approved these changes Jun 8, 2022

View reviewed changes

dprital added the Request for 202205 Branch label Jun 13, 2022

prsunny approved these changes Jun 21, 2022

View reviewed changes

prsunny merged commit 87425a5 into sonic-net:master Jun 21, 2022

yxieca removed the Request for 202205 Branch label Jun 22, 2022

liat-grozovik added the Request for 202205 Branch label Jul 6, 2022

yxieca added the Included in 202205 Branch label Jul 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sflow + dropmon] added the ENABLE_SFLOW_DROPMON build flag. Added patches for sflow repo. #10370

[sflow + dropmon] added the ENABLE_SFLOW_DROPMON build flag. Added patches for sflow repo. #10370

vadymhlushko-mlnx commented Mar 28, 2022 •

edited

Loading

prsunny commented Mar 28, 2022

vadymhlushko-mlnx commented Apr 4, 2022

mssonicbld commented Apr 4, 2022

azure-pipelines bot commented Apr 4, 2022

liat-grozovik commented Apr 22, 2022

vadymhlushko-mlnx commented May 19, 2022

mssonicbld commented May 19, 2022

azure-pipelines bot commented May 19, 2022

jeff-yin commented May 26, 2022

vadymhlushko-mlnx commented May 27, 2022 •

edited

Loading

vadymhlushko-mlnx commented Jun 7, 2022

padmanarayana commented Jun 8, 2022

jeff-yin commented Jun 8, 2022

vadymhlushko-mlnx commented Jun 8, 2022

vadymhlushko-mlnx commented Jun 10, 2022

[sflow + dropmon] added the ENABLE_SFLOW_DROPMON build flag. Added patches for sflow repo. #10370

[sflow + dropmon] added the ENABLE_SFLOW_DROPMON build flag. Added patches for sflow repo. #10370

Conversation

vadymhlushko-mlnx commented Mar 28, 2022 • edited Loading

DEPENDS ON: sonic-linux-kernel/pull/276, sonic-linux-kernel/pull/281, sonic-linux-kernel/pull/284

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

prsunny commented Mar 28, 2022

vadymhlushko-mlnx commented Apr 4, 2022

mssonicbld commented Apr 4, 2022

azure-pipelines bot commented Apr 4, 2022

liat-grozovik commented Apr 22, 2022

vadymhlushko-mlnx commented May 19, 2022

mssonicbld commented May 19, 2022

azure-pipelines bot commented May 19, 2022

jeff-yin commented May 26, 2022

vadymhlushko-mlnx commented May 27, 2022 • edited Loading

vadymhlushko-mlnx commented Jun 7, 2022

padmanarayana commented Jun 8, 2022

jeff-yin commented Jun 8, 2022

vadymhlushko-mlnx commented Jun 8, 2022

vadymhlushko-mlnx commented Jun 10, 2022

vadymhlushko-mlnx commented Mar 28, 2022 •

edited

Loading

vadymhlushko-mlnx commented May 27, 2022 •

edited

Loading