Full Mesh Traffic Sanity verification #11968

sreejithsreekumaran · 2024-03-13T10:19:34Z

Description of PR

Summary:
Fixes # (issue)

A test to check reachability from any source port to any destination port in the device for each traffic class.

Type of change

New test case to verify traffic sanity

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

Approach

What is the motivation for this PR?

How did you do it?

ptf64 topology is used to have 64 ports of the DUT connected
to 64 PTF ports.

How did you verify/test it?

Ran the test on a T1 ptf-64 topology

Any platform specific information?

This is a Cisco specific test case

Supported testbed topology if it's a new test case?

Documentation

from all src to all dst port pair. After all src inject traffic to a given dst, queue counters are verified for any drop

Fixed a bug in longest prefix match in conditional_mark Added conditional mark to skip if not ptf64 topo

alpeshspatel · 2024-03-19T11:19:58Z

@kevinwangsk FYI

kevinskwang · 2024-03-20T09:03:42Z

@sreejithsreekumaran pls upload the test results based on your branch full_mesh_traffic_sanity

sreejithsreekumaran · 2024-03-21T12:56:10Z

@sreejithsreekumaran pls upload the test results based on your branch full_mesh_traffic_sanity

Below is the snippet from the log file. I am attaching the run logs as well as the ptf logs

[WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details
collected 3 items

qos/test_qos_sai.py::TestQosSai::testQosSaiFullMeshTrafficSanity[single_asic-None] PASSED [ 33%]
qos/test_qos_sai.py::TestQosSai::testQosSaiFullMeshTrafficSanity[single_dut_multi_asic-None] SKIPPED (Did not find any frontend node that is multi-asic - s...) [ 66%]
qos/test_qos_sai.py::TestQosSai::testQosSaiFullMeshTrafficSanity[multi_dut-None] SKIPPED (Don't have 2 frontend nodes - so can't run multi_dut tests) [100%]

========================================================================== warnings summary ===========================================================================
../../usr/local/lib/python3.8/dist-packages/_yaml/init.py:18
/usr/local/lib/python3.8/dist-packages/_yaml/init.py:18: DeprecationWarning: The _yaml extension module is now located at yaml._yaml and its location is subject to change. To use the LibYAML-based parser and emitter, import from yaml: from yaml import CLoader as Loader, CDumper as Dumper.
warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------------------------------------------------- generated xml file: /run_logs/sree/tr_2024-03-21-11-21-09.xml ----------------------------------------------------
======================================================================= short test summary info =======================================================================
SKIPPED [1] qos/qos_sai_base.py:613: Did not find any frontend node that is multi-asic - so can't run single_dut_multi_asic tests
SKIPPED [1] qos/qos_sai_base.py:623: Don't have 2 frontend nodes - so can't run multi_dut tests
======================================================== 1 passed, 2 skipped, 1 warning in 4578.74s (1:16:18) =========================================================
INFO:root:Can not get Allure report URL. Please check logs

full_mesh_ptf_logs.tar.gz
fullmesh_run_log.tar.gz

sreejithsreekumaran · 2024-04-05T15:58:16Z

@kevinskwang please suggest the next step. Further, can you please resolve the conflicts?

kevinskwang · 2024-04-16T00:23:44Z

tests/common/plugins/conditional_mark/tests_mark_conditions.yaml

+  skip:
+    reason: "Unsupported testbed type."
+    conditions:
+      - "topo_name not in ['ptf64'] and asic_type in ['cisco-8000']"


From your code here, the case should be skipped if asic type is cisco-8000, how you get the test result you posted?

@kevinskwang the case will be skipped only if it's not a ptf64 topology and asic-type is cisco-8000. The test result is posted from a testbed having ptf64 topology and asic_type is cisco-8000

can this case to be used by other platforms?

yes it can be used by other platforms as long as it is from Cisco.

No, I mean other platform, like broadcom. mellanox

ok. I got it now. Yes, it's written currently for Cisco platform only.

I have fixed the condition to skip for other platforms too.

mssonicbld · 2024-05-13T09:23:49Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_qos_tests.py:1647:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1658:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1662:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1686:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1697:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1701:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1721:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1732:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1736:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1755:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1762:35: E275 missing whitespace after keyword ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

…_sanity

mssonicbld · 2024-05-13T09:35:40Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...............................................................Passed check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_qos_tests.py:1647:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1658:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1662:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1686:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1697:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1701:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1721:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1732:27: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1736:23: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1755:19: E275 missing whitespace after keyword tests/saitests/py3/sai_qos_tests.py:1762:35: E275 missing whitespace after keyword ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

sreejithsreekumaran · 2024-05-13T10:59:55Z

The pre-commit check detected issues in the files touched by this pull request. The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results: trim trailing whitespace.................................................Passedfix end of files.........................................................Passedcheck yaml...............................................................Passedcheck for added large files..............................................Passedcheck python ast.........................................................Passedflake8...................................................................Failed- hook id: flake8- exit code: 1tests/saitests/py3/sai_qos_tests.py:1647:19: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1658:27: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1662:23: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1686:19: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1697:27: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1701:23: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1721:19: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1732:27: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1736:23: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1755:19: E275 missing whitespace after keywordtests/saitests/py3/sai_qos_tests.py:1762:35: E275 missing whitespace after keyword...[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.

Ensure that the pre-commit package is installed:
sudo pip install pre-commit
Go to repository root folder

Install the pre-commit hooks:
pre-commit install
Use pre-commit to check staged file:
pre-commit
Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@kevinskwang these pre-commit warnings are from lines which are part of PR #9896. Looks like the rules have changed from then to now.

kevinskwang · 2024-05-30T05:51:51Z

could you resolve the conflicts?

sreejithsreekumaran · 2024-05-30T11:27:47Z

could you resolve the conflicts?

@kevinskwang I have resolved the conflict. Please re-approve and do the needful.

mssonicbld · 2024-09-17T00:40:07Z

@sreejithsreekumaran PR conflicts with 202311 branch

* Added testcase to run traffic in full mesh from all src to all dst port pair. After all src inject traffic to a given dst, queue counters are verified for any drop * Removed the unwanted Try block Fixed a bug in longest prefix match in conditional_mark Added conditional mark to skip if not ptf64 topo * Made adding static routes a pytest fixture * Added fixture to configure ip on ptf interfaces while using PTF64 topo * Changed to skip for all platforms except Cisco * flake8 issues reported from changes in PR 9896

mssonicbld · 2024-09-17T00:40:14Z

Cherry-pick PR to 202405: #14604

* Added testcase to run traffic in full mesh from all src to all dst port pair. After all src inject traffic to a given dst, queue counters are verified for any drop * Removed the unwanted Try block Fixed a bug in longest prefix match in conditional_mark Added conditional mark to skip if not ptf64 topo * Made adding static routes a pytest fixture * Added fixture to configure ip on ptf interfaces while using PTF64 topo * Changed to skip for all platforms except Cisco * flake8 issues reported from changes in PR 9896

sreejithsreekumaran · 2024-09-20T10:20:35Z

ptf64 topology is used to have 64 ports of the DUT connected
to 64 PTF ports.

@kevinskwang raised PR #14678

This reverts commit e15daca.

…arks (#14912) What is the motivation for this PR? As part of #11968, a bugfix was introduced in the find_longest_matches function to identify only the longest matching entry in tests_mark_conditions.yaml. The previous bug led to incorrect behavior in finding the matching entries, deviating from our intended logic before: Our expected behavior was to retrieve only the longest matching entry and evaluate the conditions under this single, unique entry. Instead, the bug caused the function to retrieve all entries matching the test case prefix, evaluating conditions from all matching entries. This resulted in an unintended situation where conditions from shorter matching entries which are absent in the longest one would still be considered during evaluation. In PR #14395, we optimized the logic for finding marks in conditional marking. Now, although we will retrieve all entries matching the test case prefix but only the longest matching entry is evaluated if the mark is identical across all matching entries, which aligns with our original expectations. Additionally, conditions that exist only in shorter matching entries with the same mark are now ignored. To maintain consistency with previous behavior(although unexpected), we have added these conditions to the longer matching entry. How did you do it? To maintain consistency with previous behavior(although unexpected), we have added these conditions to the longer matching entry.

sreejithsreekumaran added 3 commits March 13, 2024 03:14

Added testcase to run traffic in full mesh

dd76c1d

from all src to all dst port pair. After all src inject traffic to a given dst, queue counters are verified for any drop

Removed the unwanted Try block

155b2e0

Fixed a bug in longest prefix match in conditional_mark Added conditional mark to skip if not ptf64 topo

Made adding static routes a pytest fixture

d477e2c

Added fixture to configure ip on ptf interfaces while using PTF64 topo

b53bf93

rbpittman approved these changes Mar 27, 2024

View reviewed changes

kevinskwang reviewed Apr 16, 2024

View reviewed changes

sreejithsreekumaran added 2 commits April 23, 2024 10:19

Merge upstream after sync

1c46670

Changed to skip for all platforms except Cisco

2bf2987

Merge remote-tracking branch 'upstream/master' into full_mesh_traffic…

21b9a44

…_sanity

flake8 issues reported from changes in PR 9896

e45042d

kevinskwang previously approved these changes May 30, 2024

View reviewed changes

Merge master

4832ead

sreejithsreekumaran dismissed kevinskwang’s stale review via 4832ead May 30, 2024 11:26

kevinskwang added Request for 202311 branch Request for 202405 branch labels Jun 14, 2024

kevinskwang approved these changes Jun 14, 2024

View reviewed changes

kevinskwang merged commit eaba25d into sonic-net:master Jun 14, 2024
16 checks passed

kevinskwang added Approved for 202311 branch Approved for 202405 branch labels Sep 17, 2024

mssonicbld added the Cherry Pick Conflict_202311 label Sep 17, 2024

mssonicbld added the Created PR to 202405 branch label Sep 17, 2024

mssonicbld mentioned this pull request Sep 17, 2024

[action] [PR:11968] Full Mesh Traffic Sanity verification #14604

Merged

8 tasks

mssonicbld added Included in 202405 branch and removed Created PR to 202405 branch labels Sep 17, 2024

arista-nwolfe mentioned this pull request Sep 23, 2024

[Bug]: conditional_mark only considers the longest match in the yaml file even when skipping. #14698

Open

kenneth-arista added a commit to kenneth-arista/sonic-mgmt that referenced this pull request Sep 26, 2024

Revert "Full Mesh Traffic Sanity verification (sonic-net#11968)"

7c9f945

This reverts commit e15daca.

kevinskwang pushed a commit that referenced this pull request Sep 29, 2024

Revert "Full Mesh Traffic Sanity verification (#11968)" (#14754)

fbfef99

This reverts commit e15daca.

sreejithsreekumaran mentioned this pull request Oct 8, 2024

Add a suffix to the testcase logfile #14885

Open

8 tasks

yutongzhang-microsoft mentioned this pull request Oct 10, 2024

[Bugfix]Adding conditions to longer matching entries in conditional marks #14912

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full Mesh Traffic Sanity verification #11968

Full Mesh Traffic Sanity verification #11968

sreejithsreekumaran commented Mar 13, 2024 •

edited

Loading

alpeshspatel commented Mar 19, 2024

kevinskwang commented Mar 20, 2024

sreejithsreekumaran commented Mar 21, 2024

sreejithsreekumaran commented Apr 5, 2024

kevinskwang Apr 16, 2024

sreejithsreekumaran Apr 16, 2024

kevinskwang May 10, 2024

sreejithsreekumaran May 10, 2024

kevinskwang May 13, 2024

sreejithsreekumaran May 13, 2024

mssonicbld commented May 13, 2024

mssonicbld commented May 13, 2024

sreejithsreekumaran commented May 13, 2024 •

edited

Loading

kevinskwang commented May 30, 2024

sreejithsreekumaran commented May 30, 2024

mssonicbld commented Sep 17, 2024

mssonicbld commented Sep 17, 2024

sreejithsreekumaran commented Sep 20, 2024

Full Mesh Traffic Sanity verification #11968

Full Mesh Traffic Sanity verification #11968

Conversation

sreejithsreekumaran commented Mar 13, 2024 • edited Loading

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

alpeshspatel commented Mar 19, 2024

kevinskwang commented Mar 20, 2024

sreejithsreekumaran commented Mar 21, 2024

sreejithsreekumaran commented Apr 5, 2024

kevinskwang Apr 16, 2024

Choose a reason for hiding this comment

sreejithsreekumaran Apr 16, 2024

Choose a reason for hiding this comment

kevinskwang May 10, 2024

Choose a reason for hiding this comment

sreejithsreekumaran May 10, 2024

Choose a reason for hiding this comment

kevinskwang May 13, 2024

Choose a reason for hiding this comment

sreejithsreekumaran May 13, 2024

Choose a reason for hiding this comment

mssonicbld commented May 13, 2024

mssonicbld commented May 13, 2024

sreejithsreekumaran commented May 13, 2024 • edited Loading

kevinskwang commented May 30, 2024

sreejithsreekumaran commented May 30, 2024

mssonicbld commented Sep 17, 2024

mssonicbld commented Sep 17, 2024

sreejithsreekumaran commented Sep 20, 2024

sreejithsreekumaran commented Mar 13, 2024 •

edited

Loading

sreejithsreekumaran commented May 13, 2024 •

edited

Loading