Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ErrorLog] Ignore teamd portchannel and fdb error log #5951

Merged
merged 1 commit into from
Jul 13, 2022

Conversation

yejianquan
Copy link
Collaborator

Description of PR

Summary:
Use loganalyzer ignore marker to ignore 'config load' error logs.
Ignore LAG and FDB related error logs found in test case sub_port_interfaces/test_sub_port_interfaces.py::test_admin_status_down_disables_forwarding

  1. Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20'
  2. Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012

Approach

What is the motivation for this PR?

Ignore safe error logs.

How did you do it?

For error log 1, Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20'
It's raised after the test case creates some portchannel-sub-interfaces.
But the portchannel-sub-interfaces have been created successfully:

admin@foo:~/tmp/sub_port_interfaces$ show ip int
Interface        Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
---------------  --------  -------------------  ------------  --------------  -------------
Ethernet80.10              10.0.0.56/31         up/up         ARISTA01BT1     10.0.0.57
Ethernet84.10              10.0.0.58/31         up/up         ARISTA02BT1     10.0.0.59
Ethernet88.10              10.0.0.60/31         up/up         ARISTA03BT1     10.0.0.61
Ethernet92.10              10.0.0.62/31         up/up         ARISTA04BT1     10.0.0.63
Ethernet96.10              10.0.0.64/31         up/up         ARISTA05BT1     10.0.0.65
Ethernet100.10             10.0.0.66/31         up/up         ARISTA06BT1     10.0.0.67
Ethernet104.10             10.0.0.68/31         up/up         ARISTA07BT1     10.0.0.69
Ethernet108.10             10.0.0.70/31         up/up         ARISTA08BT1     10.0.0.71
Ethernet112.10             10.0.0.72/31         up/up         ARISTA09BT1     10.0.0.73
Ethernet116.10             10.0.0.74/31         up/up         ARISTA10BT1     10.0.0.75
Ethernet120.10             10.0.0.76/31         up/up         ARISTA11BT1     10.0.0.77
Ethernet124.10             10.0.0.78/31         up/up         ARISTA12BT1     10.0.0.79
Loopback0                  10.1.0.32/32         up/up         N/A             N/A
PortChannel1.20            172.16.0.1/30        up/up         N/A             N/A
PortChannel1.30            172.16.0.5/30        up/up         N/A             N/A
PortChannel1.40            172.16.0.9/30        up/up         N/A             N/A
PortChannel1.50            172.16.0.13/30       up/up         N/A             N/A
PortChannel2.20            172.16.4.1/30        up/up         N/A             N/A
PortChannel2.30            172.16.4.5/30        up/up         N/A             N/A
PortChannel2.40            172.16.4.9/30        up/up         N/A             N/A
PortChannel2.50            172.16.4.13/30       up/up         N/A             N/A
Vlan1000                   192.168.0.1/21       up/up         N/A             N/A
docker0                    240.127.1.1/24       up/down       N/A             N/A
eth0                       10.64.247.28/23      up/up         N/A             N/A
lo                         127.0.0.1/16         up/up         N/A             N/A

admin@foo:~/tmp/sub_port_interfaces$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.095 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 172.16.0.1: icmp_seq=3 ttl=64 time=0.124 ms

According to: sonic-net/sonic-buildimage#11218
I think we can safely ignore it in the sub port test case.

For error log 2, Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f.
It's raised after removing a member interface from the vlan, because the deleted interface loses its fdb entry.

As for the log analyzer ignore marker in the 'config load', I've to do that slight refactor, otherwise we can't only ignore 2 patterns in ignore_regex_list, but also include all error logs caused by 'config load'

How did you verify/test it?

Run it on physical testbed:

=== Running tests in groups ===
/usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
================================================================================================================== test session starts ===================================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.5, py-1.10.0, pluggy-0.13.1
ansible: 2.8.12
rootdir: /var/src/sonic-mgmt-int/tests, inifile: pytest.ini
plugins: celery-4.4.7, forked-1.3.0, metadata-1.11.0, xdist-1.28.0, html-1.22.1, repeat-0.9.1, ansible-2.2.2
collecting ... /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
collected 1 item                                                                                                                                                                                                                                         

sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_admin_status_down_disables_forwarding[port_in_lag] PASSED                                                                                                                      [100%]

--------------------------------------------------------------------------------------------- generated xml file: /var/src/sonic-mgmt-int/tests/logs/tr.xml ----------------------------------------------------------------------------------------------
=============================================================================================================== 1 passed in 582.00 seconds ===============================================================================================================

@yejianquan yejianquan requested a review from a team as a code owner July 8, 2022 09:36
@yejianquan
Copy link
Collaborator Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@yejianquan
Copy link
Collaborator Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@neethajohn neethajohn merged commit 4f1cee7 into sonic-net:master Jul 13, 2022
yejianquan added a commit that referenced this pull request Jul 18, 2022
Summary:
Use loganalyzer ignore marker to ignore 'config load' error logs.
Ignore LAG and FDB related error logs found in test case sub_port_interfaces/test_sub_port_interfaces.py::test_admin_status_down_disables_forwarding

Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20'
Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f.

How did you do it?
For error log 1, Jul 8 07:00:39.302949 foo-DUT ERR teamd#tlm_teamd: :- process_add_queue: Can't connect to teamd after 10 attempts. LAG 'PortChannel1.20'
It's raised after the test case creates some portchannel-sub-interfaces.
But the portchannel-sub-interfaces have been created successfully:

admin@foo:~/tmp/sub_port_interfaces$ show ip int
Interface        Master    IPv4 address/mask    Admin/Oper    BGP Neighbor    Neighbor IP
---------------  --------  -------------------  ------------  --------------  -------------
Ethernet80.10              10.0.0.56/31         up/up         ARISTA01BT1     10.0.0.57
Ethernet84.10              10.0.0.58/31         up/up         ARISTA02BT1     10.0.0.59
Ethernet88.10              10.0.0.60/31         up/up         ARISTA03BT1     10.0.0.61
Ethernet92.10              10.0.0.62/31         up/up         ARISTA04BT1     10.0.0.63
Ethernet96.10              10.0.0.64/31         up/up         ARISTA05BT1     10.0.0.65
Ethernet100.10             10.0.0.66/31         up/up         ARISTA06BT1     10.0.0.67
Ethernet104.10             10.0.0.68/31         up/up         ARISTA07BT1     10.0.0.69
Ethernet108.10             10.0.0.70/31         up/up         ARISTA08BT1     10.0.0.71
Ethernet112.10             10.0.0.72/31         up/up         ARISTA09BT1     10.0.0.73
Ethernet116.10             10.0.0.74/31         up/up         ARISTA10BT1     10.0.0.75
Ethernet120.10             10.0.0.76/31         up/up         ARISTA11BT1     10.0.0.77
Ethernet124.10             10.0.0.78/31         up/up         ARISTA12BT1     10.0.0.79
Loopback0                  10.1.0.32/32         up/up         N/A             N/A
PortChannel1.20            172.16.0.1/30        up/up         N/A             N/A
PortChannel1.30            172.16.0.5/30        up/up         N/A             N/A
PortChannel1.40            172.16.0.9/30        up/up         N/A             N/A
PortChannel1.50            172.16.0.13/30       up/up         N/A             N/A
PortChannel2.20            172.16.4.1/30        up/up         N/A             N/A
PortChannel2.30            172.16.4.5/30        up/up         N/A             N/A
PortChannel2.40            172.16.4.9/30        up/up         N/A             N/A
PortChannel2.50            172.16.4.13/30       up/up         N/A             N/A
Vlan1000                   192.168.0.1/21       up/up         N/A             N/A
docker0                    240.127.1.1/24       up/down       N/A             N/A
eth0                       10.64.247.28/23      up/up         N/A             N/A
lo                         127.0.0.1/16         up/up         N/A             N/A

admin@foo:~/tmp/sub_port_interfaces$ ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.095 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 172.16.0.1: icmp_seq=3 ttl=64 time=0.124 ms
According to: sonic-net/sonic-buildimage#11218
I think we can safely ignore it in the sub port test case.

For error log 2, Jul 8 07:28:02.105624 foo-DUT ERR swss#orchagent: :- update: Failed to get port by bridge port ID 0x3a00000000083f.
It's raised after removing a member interface from the vlan, because the deleted interface loses its fdb entry.

As for the log analyzer ignore marker in the 'config load', I've to do that slight refactor, otherwise we can't only ignore 2 patterns in ignore_regex_list, but also include all error logs caused by 'config load'

How did you verify/test it?
Run it on physical testbed:

=== Running tests in groups ===
/usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
================================================================================================================== test session starts ===================================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.5, py-1.10.0, pluggy-0.13.1
ansible: 2.8.12
rootdir: /var/src/sonic-mgmt-int/tests, inifile: pytest.ini
plugins: celery-4.4.7, forked-1.3.0, metadata-1.11.0, xdist-1.28.0, html-1.22.1, repeat-0.9.1, ansible-2.2.2
collecting ... /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature
collected 1 item                                                                                                                                                                                                                                         

sub_port_interfaces/test_sub_port_interfaces.py::TestSubPorts::test_admin_status_down_disables_forwarding[port_in_lag] PASSED                                                                                                                      [100%]

--------------------------------------------------------------------------------------------- generated xml file: /var/src/sonic-mgmt-int/tests/logs/tr.xml ----------------------------------------------------------------------------------------------
=============================================================================================================== 1 passed in
neethajohn pushed a commit that referenced this pull request Aug 19, 2022
PR #5951 introduced 2 error log ignore while running sub_port_interfaces test suite. Sub port folder has 2 more cases that need those skip to be performed

What is the motivation for this PR?
Enable ignore for all sub port test suite

How did you do it?
Move ignore fixture from test to conftest scope

How did you verify/test it?
Run TC, all passed
allen-xf pushed a commit to allen-xf/sonic-mgmt that referenced this pull request Oct 28, 2022
)

PR sonic-net#5951 introduced 2 error log ignore while running sub_port_interfaces test suite. Sub port folder has 2 more cases that need those skip to be performed

What is the motivation for this PR?
Enable ignore for all sub port test suite

How did you do it?
Move ignore fixture from test to conftest scope

How did you verify/test it?
Run TC, all passed
wangxin pushed a commit that referenced this pull request Nov 17, 2022
PR #5951 introduced 2 error log ignore while running sub_port_interfaces test suite. Sub port folder has 2 more cases that need those skip to be performed

What is the motivation for this PR?
Enable ignore for all sub port test suite

How did you do it?
Move ignore fixture from test to conftest scope

How did you verify/test it?
Run TC, all passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants