Adding COUNTER_MARGIN to Ingress drop counters for qos tests #9896

ansrajpu-git · 2023-09-07T16:46:58Z

Description of PR

Adding COUNTER_MARGIN to ingress drop counter for below test scenarios to avoid flakey results.

TestQosSai.testQosSaiPfcXoffLimit
TestQosSai.testQosSaiPfcXonLimit
TestQosSai.testQosSaiLossyQueue

Summary:
Sometimes for dnx, few extra ipv6 NS/RA pkt received from VM which get added to counter value & may give inconsistent test results.
Adding COUNTER_MARGIN to provide room to 2 extra pkt incase, extra traffic received. This will avoid inconsistent results on qos test.
Below tests are updated:

TestQosSai.testQosSaiPfcXoffLimit
TestQosSai.testQosSaiPfcXonLimit
TestQosSai.testQosSaiLossyQueue
Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012
202205

Approach

Sometimes for dnx, few extra ipv6 NS/RA pkt received from VM which get added to counter value & may give inconsistent test results.
Providing room for 2 pkts (COUNTER_MARGIN) incase, extra traffic received.
Below test are updated:

TestQosSai.testQosSaiPfcXoffLimit
TestQosSai.testQosSaiPfcXonLimit
TestQosSai.testQosSaiLossyQueue

What is the motivation for this PR?

Inconsistent results on qos test runs

How did you do it?

How did you verify/test it?

Executed qos tests & verify the results.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld · 2023-09-07T16:48:03Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_qos_tests.py:1314:121: E501 line too long (125 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1347:121: E501 line too long (125 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1381:121: E501 line too long (125 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1414:121: E501 line too long (125 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1419:21: E122 continuation line missing indentation or outdented tests/saitests/py3/sai_qos_tests.py:1421:121: E501 line too long (142 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1424:121: E501 line too long (135 > 120 characters) tests/saitests/py3/sai_qos_tests.py:1859:9: F841 local variable 'max_buffer_size' is assigned to but never used tests/saitests/py3/sai_qos_tests.py:2135:121: E501 line too long (125 > 120 characters) tests/saitests/py3/sai_qos_tests.py:3361:26: E211 whitespace before '(' tests/saitests/py3/sai_qos_tests.py:3361:73: E225 missing whitespace around operator ... [truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

vmittal-msft · 2023-09-13T22:07:44Z

tests/saitests/py3/sai_qos_tests.py

-                for cntr in egress_counters:
-                    assert (xmit_counters[cntr] == xmit_counters_base[cntr]
-                            ), 'unexpectedly TX drop counter increase, {}'.format(test_stage)
+            # For dnx few extra ipv6 NS/RA pkt received from VM, adding to counter value


@ansrajpu-git Original code is stating these counters not supported for DNX. Is this not the case ?

@ansrajpu-git Original code is stating these counters not supported for DNX. Is this not the case ?

vineet, these drop counters were not supported initially and they are supported now. So enabling the check

tests/saitests/py3/sai_qos_tests.py

mssonicbld · 2023-09-20T22:44:33Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/saitests/py3/sai_qos_tests.py:3393:30: E211 whitespace before '(' tests/saitests/py3/sai_qos_tests.py:3393:77: E225 missing whitespace around operator tests/saitests/py3/sai_qos_tests.py:3393:121: E501 line too long (140 > 120 characters) tests/saitests/py3/sai_qos_tests.py:3393:124: E231 missing whitespace after ',' flake8...............................................(no files to check)Skipped check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

judyjoseph · 2023-09-29T20:45:37Z

@ansrajpu-git Could you raise a separate PR 202205 branch ? as there are conflicts

…for qos tests (sonic-net#9896) * Qos_LossyQueueTest fix_adding COUNTER MARGIN * Adding COUNTER_MARGIN to qos PFCXonTest * Adding COUNTER_MARGIN to qos PFCTest * Flake8 fixes * testQosSaiPFCXoffLimit updated with correct check * Conditional check corrected for LossyQueueTest * flake8 fix

…for qos tests (#9896) (#10210) Backport master branch PR #9896

…et#9896) * Qos_LossyQueueTest fix_adding COUNTER MARGIN * Adding COUNTER_MARGIN to qos PFCXonTest * Adding COUNTER_MARGIN to qos PFCTest * Flake8 fixes * testQosSaiPFCXoffLimit updated with correct check * Conditional check corrected for LossyQueueTest * flake8 fix

mssonicbld · 2023-10-19T02:57:20Z

Cherry-pick PR to 202305: #10390

* Qos_LossyQueueTest fix_adding COUNTER MARGIN * Adding COUNTER_MARGIN to qos PFCXonTest * Adding COUNTER_MARGIN to qos PFCTest * Flake8 fixes * testQosSaiPFCXoffLimit updated with correct check * Conditional check corrected for LossyQueueTest * flake8 fix

…et#9896) * Qos_LossyQueueTest fix_adding COUNTER MARGIN * Adding COUNTER_MARGIN to qos PFCXonTest * Adding COUNTER_MARGIN to qos PFCTest * Flake8 fixes * testQosSaiPFCXoffLimit updated with correct check * Conditional check corrected for LossyQueueTest * flake8 fix

ansrajpu-git added 3 commits September 7, 2023 11:16

Qos_LossyQueueTest fix_adding COUNTER MARGIN

9dfe934

Adding COUNTER_MARGIN to qos PFCXonTest

452c74b

Adding COUNTER_MARGIN to qos PFCTest

dde1636

Flake8 fixes

52c6ab5

vmittal-msft reviewed Sep 13, 2023

View reviewed changes

vmittal-msft added the Request for 202205 branch label Sep 13, 2023

vmittal-msft reviewed Sep 15, 2023

View reviewed changes

tests/saitests/py3/sai_qos_tests.py Show resolved Hide resolved

rlhui assigned ansrajpu-git Sep 20, 2023

ansrajpu-git added 2 commits September 20, 2023 17:57

testQosSaiPFCXoffLimit updated with correct check

af48739

Conditional check corrected for LossyQueueTest

dba1964

flake8 fix

21b6d31

judyjoseph approved these changes Sep 25, 2023

View reviewed changes

vmittal-msft approved these changes Sep 25, 2023

View reviewed changes

vmittal-msft merged commit a7be8e0 into sonic-net:master Sep 25, 2023
12 checks passed

ansrajpu-git mentioned this pull request Oct 2, 2023

[Cherry-pick] Adding COUNTER_MARGIN to Ingress drop counters …#9896 #10210

Merged

6 tasks

wangxin pushed a commit that referenced this pull request Oct 19, 2023

Cherry pick to 202205 Adding COUNTER_MARGIN to Ingress drop counters …

4adc032

…for qos tests (#9896) (#10210) Backport master branch PR #9896

wangxin added Included in 202205 branch Request for 202305 branch Approved for 202305 branch labels Oct 19, 2023

mssonicbld added the Created PR to 202305 branch label Oct 19, 2023

mssonicbld mentioned this pull request Oct 19, 2023

[action] [PR:9896] Adding COUNTER_MARGIN to Ingress drop counters for qos tests #10390

Merged

6 tasks

mssonicbld added Included in 202305 branch and removed Created PR to 202305 branch labels Oct 20, 2023

sreejithsreekumaran mentioned this pull request May 13, 2024

Full Mesh Traffic Sanity verification #11968

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding COUNTER_MARGIN to Ingress drop counters for qos tests #9896

Adding COUNTER_MARGIN to Ingress drop counters for qos tests #9896

ansrajpu-git commented Sep 7, 2023

mssonicbld commented Sep 7, 2023

vmittal-msft Sep 13, 2023

saksarav-nokia Sep 15, 2023

mssonicbld commented Sep 20, 2023

judyjoseph commented Sep 29, 2023

mssonicbld commented Oct 19, 2023

Adding COUNTER_MARGIN to Ingress drop counters for qos tests #9896

Adding COUNTER_MARGIN to Ingress drop counters for qos tests #9896

Conversation

ansrajpu-git commented Sep 7, 2023

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld commented Sep 7, 2023

vmittal-msft Sep 13, 2023

Choose a reason for hiding this comment

saksarav-nokia Sep 15, 2023

Choose a reason for hiding this comment

mssonicbld commented Sep 20, 2023

judyjoseph commented Sep 29, 2023

mssonicbld commented Oct 19, 2023