From 055fbf5aaaf4b8269cd598931f2c3fc3a41987cd Mon Sep 17 00:00:00 2001 From: Samuel Angebault Date: Wed, 14 Sep 2022 04:39:49 +0200 Subject: [PATCH 01/51] [Arista] Update platform submodules (#12020) --- platform/barefoot/sonic-platform-modules-arista | 2 +- platform/broadcom/sonic-platform-modules-arista | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/platform/barefoot/sonic-platform-modules-arista b/platform/barefoot/sonic-platform-modules-arista index b65a69a9e1c2..e12a04b24c5f 160000 --- a/platform/barefoot/sonic-platform-modules-arista +++ b/platform/barefoot/sonic-platform-modules-arista @@ -1 +1 @@ -Subproject commit b65a69a9e1c2c876ba5210ce8b2a1cc9b5c8b18f +Subproject commit e12a04b24c5f752a9ca789d62bb7b94c563e1c4b diff --git a/platform/broadcom/sonic-platform-modules-arista b/platform/broadcom/sonic-platform-modules-arista index b65a69a9e1c2..e12a04b24c5f 160000 --- a/platform/broadcom/sonic-platform-modules-arista +++ b/platform/broadcom/sonic-platform-modules-arista @@ -1 +1 @@ -Subproject commit b65a69a9e1c2c876ba5210ce8b2a1cc9b5c8b18f +Subproject commit e12a04b24c5f752a9ca789d62bb7b94c563e1c4b From c674b3c63486082a0d426ae64e58f70480205ddc Mon Sep 17 00:00:00 2001 From: kannankvs Date: Wed, 14 Sep 2022 10:19:05 +0530 Subject: [PATCH 02/51] [doc]: Updated PR Template for a comment to add label/tag for the feature raised. (#12058) Signed-off-by: kannankvs --- .github/pull_request_template.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 1ff6883573d2..d553f7a4d0c4 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -39,6 +39,8 @@ Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> +#### Ensure to add label/tag for the feature raised. example - [PR#2174](https://github.com/sonic-net/sonic-utilities/pull/2174) where, Generic Config and Update feature has been labelled as GCU. + #### Link to config_db schema for YANG module changes - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 #### Description for the changelog Improve SSHD config to use more secure settings #### Link to config_db schema for YANG module changes #### A picture of a cute animal (not mandatory but encouraged) --- build_debian.sh | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/build_debian.sh b/build_debian.sh index 8dcd8596f684..01a10d78d678 100755 --- a/build_debian.sh +++ b/build_debian.sh @@ -467,10 +467,16 @@ rm /files/etc/ssh/sshd_config/ClientAliveInterval rm /files/etc/ssh/sshd_config/ClientAliveCountMax touch /files/etc/ssh/sshd_config/EmptyLineHack rename /files/etc/ssh/sshd_config/EmptyLineHack "" -set /files/etc/ssh/sshd_config/ClientAliveInterval 900 +set /files/etc/ssh/sshd_config/ClientAliveInterval 300 set /files/etc/ssh/sshd_config/ClientAliveCountMax 1 ins #comment before /files/etc/ssh/sshd_config/ClientAliveInterval -set /files/etc/ssh/sshd_config/#comment[following-sibling::*[1][self::ClientAliveInterval]] "Close inactive client sessions after 15 minutes" +set /files/etc/ssh/sshd_config/#comment[following-sibling::*[1][self::ClientAliveInterval]] "Close inactive client sessions after 5 minutes" +rm /files/etc/ssh/sshd_config/MaxAuthTries +set /files/etc/ssh/sshd_config/MaxAuthTries 3 +rm /files/etc/ssh/sshd_config/LogLevel +set /files/etc/ssh/sshd_config/LogLevel VERBOSE +rm /files/etc/ssh/sshd_config/Banner +set /files/etc/ssh/sshd_config/Banner /etc/issue save quit EOF From 8af369a7c94d461338ee0aa63202b11252742e2c Mon Sep 17 00:00:00 2001 From: Xichen96 Date: Thu, 22 Sep 2022 13:57:52 +0800 Subject: [PATCH 20/51] Enable swap for haliburton device. (#11746) Signed-off-by: Xichen Lin Signed-off-by: Xichen Lin --- .../debian/platform-modules-haliburton.init | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/platform/broadcom/sonic-platform-modules-cel/debian/platform-modules-haliburton.init b/platform/broadcom/sonic-platform-modules-cel/debian/platform-modules-haliburton.init index 07a6542109b5..3f6358bf4ab4 100644 --- a/platform/broadcom/sonic-platform-modules-cel/debian/platform-modules-haliburton.init +++ b/platform/broadcom/sonic-platform-modules-cel/debian/platform-modules-haliburton.init @@ -11,6 +11,26 @@ # Short-Description: Setup Haliburton board. ### END INIT INFO +setup_swap () { + SWAPFILE=/host/myswapfile + + if [ ! -f $SWAPFILE ]; then + availspace=`df -h --output=avail /host | sed '1d;s/\s//g;s/[^0-9].*//g'` + diff=$(( availspace - 2*$1 )) + if [ $diff -gt 0 ]; then + fallocate -l ${1}G $SWAPFILE + chmod 600 $SWAPFILE + echo "swap file created successfully" + else + echo "not enough disk space to turn on swap." + return + fi + fi + mkswap $SWAPFILE + swapon $SWAPFILE + echo "swap on successfully" +} + case "$1" in start) echo -n "Setting up board... " @@ -74,6 +94,8 @@ start) /bin/sh /usr/local/bin/platform_api_mgnt.sh init + setup_swap 2 + echo "done." ;; From 283efeeaccd386f99e3f9242f54d76bfd2901f44 Mon Sep 17 00:00:00 2001 From: Mai Bui Date: Thu, 22 Sep 2022 06:40:42 -0700 Subject: [PATCH 21/51] [sonic-py-common] Add getstatusoutput_noshell() functions to general module (#12065) Signed-off-by: maipbui #### Why I did it `getstatusoutput()` function from `subprocess` module has shell injection issue because it includes `shell=True` in the implementation Eliminate duplicate code #### How I did it Reimplement `getstatusoutput_noshell()` and `getstatusoutput_noshell_pipe()` functions with `shell=False` Add `check_output_pipe()` function #### How to verify it Pass UT --- .../sonic_py_common/general.py | 68 +++++++++++++++++++ src/sonic-py-common/tests/test_general.py | 31 +++++++++ 2 files changed, 99 insertions(+) create mode 100644 src/sonic-py-common/tests/test_general.py diff --git a/src/sonic-py-common/sonic_py_common/general.py b/src/sonic-py-common/sonic_py_common/general.py index 9e04f3e214ee..4fea165de9fb 100644 --- a/src/sonic-py-common/sonic_py_common/general.py +++ b/src/sonic-py-common/sonic_py_common/general.py @@ -1,4 +1,5 @@ import sys +from subprocess import Popen, STDOUT, PIPE, CalledProcessError, check_output def load_module_from_source(module_name, file_path): @@ -23,3 +24,70 @@ def load_module_from_source(module_name, file_path): sys.modules[module_name] = module return module + + +def getstatusoutput_noshell(cmd): + """ + This function implements getstatusoutput API from subprocess module + but using shell=False to prevent shell injection. + Ref: https://github.com/python/cpython/blob/3.10/Lib/subprocess.py#L602 + """ + try: + output = check_output(cmd, universal_newlines=True, stderr=STDOUT) + exitcode = 0 + except CalledProcessError as ex: + output = ex.output + exitcode = ex.returncode + if output[-1:] == '\n': + output = output[:-1] + return exitcode, output + + +def getstatusoutput_noshell_pipe(cmd0, *args): + """ + This function implements getstatusoutput API from subprocess module + but using shell=False to prevent shell injection. Input command + includes two or more commands connected by shell pipe(s). + """ + popens = [Popen(cmd0, stdout=PIPE, universal_newlines=True)] + i = 0 + while i < len(args): + popens.append(Popen(args[i], stdin=popens[i].stdout, stdout=PIPE, universal_newlines=True)) + popens[i].stdout.close() + i += 1 + output = popens[-1].communicate()[0] + if output[-1:] == '\n': + output = output[:-1] + + exitcodes = [0] * len(popens) + while popens: + last = popens.pop(-1) + exitcodes[len(popens)] = last.wait() + + return (exitcodes, output) + + +def check_output_pipe(cmd0, *args): + """ + This function implements check_output API from subprocess module. + Input command includes two or more commands connected by shell pipe(s) + """ + popens = [Popen(cmd0, stdout=PIPE, universal_newlines=True)] + i = 0 + while i < len(args): + popens.append(Popen(args[i], stdin=popens[i].stdout, stdout=PIPE, universal_newlines=True)) + popens[i].stdout.close() + i += 1 + output = popens[-1].communicate()[0] + + i = 0 + args_list = [cmd0] + list(args) + while popens: + current = popens.pop(0) + exitcode = current.wait() + if exitcode != 0: + raise CalledProcessError(returncode=exitcode, cmd=args_list[i], output=current.stdout) + i += 1 + + return output + diff --git a/src/sonic-py-common/tests/test_general.py b/src/sonic-py-common/tests/test_general.py new file mode 100644 index 000000000000..a395cf9aeb6b --- /dev/null +++ b/src/sonic-py-common/tests/test_general.py @@ -0,0 +1,31 @@ +import sys +import pytest +import subprocess +from sonic_py_common.general import getstatusoutput_noshell, getstatusoutput_noshell_pipe, check_output_pipe + + +def test_getstatusoutput_noshell(tmp_path): + exitcode, output = getstatusoutput_noshell(['echo', 'sonic']) + assert (exitcode, output) == (0, 'sonic') + + exitcode, output = getstatusoutput_noshell([sys.executable, "-c", "import sys; sys.exit(6)"]) + assert exitcode != 0 + +def test_getstatusoutput_noshell_pipe(): + exitcode, output = getstatusoutput_noshell_pipe(['echo', 'sonic'], ['awk', '{print $1}']) + assert (exitcode, output) == ([0, 0], 'sonic') + + exitcode, output = getstatusoutput_noshell_pipe([sys.executable, "-c", "import sys; sys.exit(6)"], [sys.executable, "-c", "import sys; sys.exit(8)"]) + assert exitcode == [6, 8] + +def test_check_output_pipe(): + output = check_output_pipe(['echo', 'sonic'], ['awk', '{print $1}']) + assert output == 'sonic\n' + + with pytest.raises(subprocess.CalledProcessError) as e: + check_output_pipe([sys.executable, "-c", "import sys; sys.exit(6)"], [sys.executable, "-c", "import sys; sys.exit(0)"]) + assert e.returncode == [6, 0] + + with pytest.raises(subprocess.CalledProcessError) as e: + check_output_pipe([sys.executable, "-c", "import sys; sys.exit(0)"], [sys.executable, "-c", "import sys; sys.exit(6)"]) + assert e.returncode == [0, 6] From 57ff7a230827102f2d04cfdae8c3fcf242b09b50 Mon Sep 17 00:00:00 2001 From: "Marty Y. Lok" <76118573+mlok-nokia@users.noreply.github.com> Date: Thu, 22 Sep 2022 19:39:31 -0400 Subject: [PATCH 22/51] [chassis][supervisor] show system-health summary fails on the supervisor card (#10631) Fix the command "sudo show system-health summary" shows the following error on the supervisor card. Fixes #10630 --- src/system-health/health_checker/service_checker.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/system-health/health_checker/service_checker.py b/src/system-health/health_checker/service_checker.py index 33eeb4dfb707..674ad9944280 100644 --- a/src/system-health/health_checker/service_checker.py +++ b/src/system-health/health_checker/service_checker.py @@ -127,11 +127,11 @@ def get_critical_process_list_from_file(self, container, critical_processes_file self.bad_containers.add(container) logger.log_error('Invalid syntax in critical_processes file of {}'.format(container)) continue - - identifier_key = match.group(2).strip() - identifier_value = match.group(3).strip() - if identifier_key == "program" and identifier_value: - critical_process_list.append(identifier_value) + if match.group(1) is not None: + identifier_key = match.group(2).strip() + identifier_value = match.group(3).strip() + if identifier_key == "program" and identifier_value: + critical_process_list.append(identifier_value) return critical_process_list From 27032bfb84de0db2733dd024e54f0f0f83a0c769 Mon Sep 17 00:00:00 2001 From: Samuel Angebault Date: Fri, 23 Sep 2022 02:52:40 +0200 Subject: [PATCH 23/51] Add BUILD_DATE to SWI (#11915) Add the BUILD_DATE to the SWI version info, as this is a requirement of Secure Boot. --- build_image.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/build_image.sh b/build_image.sh index ddf134e845ce..9c70713866a4 100755 --- a/build_image.sh +++ b/build_image.sh @@ -191,6 +191,7 @@ elif [ "$IMAGE_TYPE" = "aboot" ]; then zip -g $ABOOT_BOOT_IMAGE .imagehash rm .imagehash echo "SWI_VERSION=42.0.0" > version + echo "BUILD_DATE=$(date -d "${build_date}" -u +%Y%m%dT%H%M%SZ)" >> version echo "SWI_MAX_HWEPOCH=2" >> version echo "SWI_VARIANT=US" >> version zip -g $OUTPUT_ABOOT_IMAGE version From c968114a36dacd46b3dfea0e61b52a72daac4737 Mon Sep 17 00:00:00 2001 From: Liu Shilong Date: Fri, 23 Sep 2022 12:54:57 +0800 Subject: [PATCH 24/51] [ci] Use absolute template file path in docker-sonic-slave pipeline. (#12153) --- .azure-pipelines/docker-sonic-slave-template.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.azure-pipelines/docker-sonic-slave-template.yml b/.azure-pipelines/docker-sonic-slave-template.yml index 1a7b983b5c1f..142ff5fa3f6f 100644 --- a/.azure-pipelines/docker-sonic-slave-template.yml +++ b/.azure-pipelines/docker-sonic-slave-template.yml @@ -38,12 +38,12 @@ jobs: - job: Build_${{ parameters.dist }}_${{ parameters.march }}${{ parameters.arch }} timeoutInMinutes: 360 variables: - - template: .azure-pipelines/template-variables.yml@buildimage - - template: .azure-pipelines/azure-pipelines-repd-build-variables.yml@buildimage + - template: /.azure-pipelines/template-variables.yml@buildimage + - template: /.azure-pipelines/azure-pipelines-repd-build-variables.yml@buildimage pool: ${{ parameters.pool }} steps: - template: cleanup.yml - - template: .azure-pipelines/template-clean-sonic-slave.yml@buildimage + - template: /.azure-pipelines/template-clean-sonic-slave.yml@buildimage - checkout: self clean: true submodules: recursive @@ -62,7 +62,7 @@ jobs: exit 0 fi - DOCKER_DATA_ROOT_FOR_MULTIARCH=/data/march/docker make configure PLATFORM=generic PLATFORM_ARCH=${{ parameters.arch }} $args || docker image ls $image_tag + DOCKER_DATA_ROOT_FOR_MULTIARCH=/data/march/docker BLDENV=${{ parameters.dist }} make -f Makefile.work configure PLATFORM=generic PLATFORM_ARCH=${{ parameters.arch }} $args || docker image ls $image_tag if [[ "$(Build.Reason)" == "PullRequest" ]];then exit 0 fi From cc0781b40b7e442677eb4ad21d72a29f16a666b3 Mon Sep 17 00:00:00 2001 From: Hua Liu <58683130+liuh-80@users.noreply.github.com> Date: Sun, 25 Sep 2022 03:37:35 +0800 Subject: [PATCH 25/51] Build swss-common with libyang (#12087) Build swss-common with libyang #### Why I did it sonic-swss-common lib add dependency to libyang recently, so need update make file before update sonic-swss-common submodule. #### How I did it Add dependency to libyang in rules/swss-common.mk #### How to verify it Pass all E2E test case. #### Which release branch to backport (provide reason below if selected) - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 #### Description for the changelog Add new Redis database PROFILE_DB #### Link to config_db schema for YANG module changes #### A picture of a cute animal (not mandatory but encouraged) --- rules/swss-common.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rules/swss-common.mk b/rules/swss-common.mk index 9ab49b672b6c..d2c4390ef4ff 100644 --- a/rules/swss-common.mk +++ b/rules/swss-common.mk @@ -9,9 +9,9 @@ $(LIBSWSSCOMMON)_VERSION = $(LIBSWSSCOMMON_VERSION) $(LIBSWSSCOMMON)_NAME = $(LIBSWSSCOMMON_NAME) $(LIBSWSSCOMMON)_DEPENDS += $(LIBHIREDIS_DEV) $(LIBNL3_DEV) $(LIBNL_GENL3_DEV) \ $(LIBNL_ROUTE3_DEV) $(LIBNL_NF3_DEV) \ - $(LIBNL_CLI_DEV) + $(LIBNL_CLI_DEV) $(LIBYANG_DEV) $(LIBYANG) $(LIBSWSSCOMMON)_RDEPENDS += $(LIBHIREDIS) $(LIBNL3) $(LIBNL_GENL3) \ - $(LIBNL_ROUTE3) $(LIBNL_NF3) $(LIBNL_CLI) + $(LIBNL_ROUTE3) $(LIBNL_NF3) $(LIBNL_CLI) $(LIBYANG) SONIC_DPKG_DEBS += $(LIBSWSSCOMMON) LIBSWSSCOMMON_DEV = $(LIBSWSSCOMMON_NAME)-dev_$(LIBSWSSCOMMON_VERSION)_$(CONFIGURED_ARCH).deb From f50dc28789cd2fdec99131284cc4ccc5192b3806 Mon Sep 17 00:00:00 2001 From: Xin Wang Date: Mon, 26 Sep 2022 10:48:02 +0800 Subject: [PATCH 26/51] [docker-sonic-mgmt] Deprecate azure-kusto-data & azure-kusto-ingest for py2 (#12143) Why I did it The python packages azure-kusto-data and azure-kusto-ingest packages for python2 are too old and not really used. The python3 environment has newer version of these packages installed. This change is to deprecate these two packages for python2 in docker-sonic-mgmt image. How I did it Removed the lines for installing old version of packages azure-kusto-data and azure-kusto-ingest in python2 in the Dockerfile template. Signed-off-by: Xin Wang --- dockers/docker-sonic-mgmt/Dockerfile.j2 | 4 ---- 1 file changed, 4 deletions(-) diff --git a/dockers/docker-sonic-mgmt/Dockerfile.j2 b/dockers/docker-sonic-mgmt/Dockerfile.j2 index 42aaa3525a64..fec662dff0ed 100755 --- a/dockers/docker-sonic-mgmt/Dockerfile.j2 +++ b/dockers/docker-sonic-mgmt/Dockerfile.j2 @@ -114,10 +114,6 @@ RUN apt-get update \ # Install Azure CLI RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash -# Install Microsoft Azure Kusto Library for Python -RUN pip install azure-kusto-data==0.0.13 \ - azure-kusto-ingest==0.0.13 - RUN pip install wheel==0.33.6 ## Copy and install sonic-mgmt docker dependencies From 2c10ebb4fe978bded6499365d3471e8343573d8c Mon Sep 17 00:00:00 2001 From: Aryeh Feigin <101218333+arfeigin@users.noreply.github.com> Date: Mon, 26 Sep 2022 19:01:49 +0300 Subject: [PATCH 27/51] Use warm-boot infrastructure for fast-boot (#11594) This PR should be merged together with the sonic-utilities PR (sonic-net/sonic-utilities#2286) and sonic-sairedis PR (sonic-net/sonic-sairedis#1100). Use redis contents from dump file in fast-reboot. Improve fast-reboot flow by utilizing the warm-reboot infrastructure. This followes https://github.com/sonic-net/SONiC/blob/master/doc/fast-reboot/Fast-reboot_Flow_Improvements_HLD.md --- files/build_templates/docker_image_ctl.j2 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/files/build_templates/docker_image_ctl.j2 b/files/build_templates/docker_image_ctl.j2 index b4ff4a4379c9..ace39df4e546 100644 --- a/files/build_templates/docker_image_ctl.j2 +++ b/files/build_templates/docker_image_ctl.j2 @@ -87,7 +87,7 @@ function preStartAction() {%- if docker_container_name == "database" %} WARM_DIR=/host/warmboot if [ "$DATABASE_TYPE" != "chassisdb" ]; then - if [[ ("$BOOT_TYPE" == "warm" || "$BOOT_TYPE" == "fastfast") && -f $WARM_DIR/dump.rdb ]]; then + if [[ ("$BOOT_TYPE" == "warm" || "$BOOT_TYPE" == "fastfast" || "$BOOT_TYPE" == "fast") && -f $WARM_DIR/dump.rdb ]]; then # Load redis content from /host/warmboot/dump.rdb docker cp $WARM_DIR/dump.rdb database$DEV:/var/lib/redis/dump.rdb else @@ -218,7 +218,7 @@ function postStartAction() ($(docker exec -i database$DEV sonic-db-cli PING | grep -c PONG) -gt 0) ]]; do sleep 1; done - if [[ ("$BOOT_TYPE" == "warm" || "$BOOT_TYPE" == "fastfast") && -f $WARM_DIR/dump.rdb ]]; then + if [[ ("$BOOT_TYPE" == "warm" || "$BOOT_TYPE" == "fastfast" || "$BOOT_TYPE" == "fast") && -f $WARM_DIR/dump.rdb ]]; then # retain the dump file from last boot for debugging purposes mv $WARM_DIR/dump.rdb $WARM_DIR/dump.rdb.old else From 1b50a2b72111593af51906f9bcaea89f1f763126 Mon Sep 17 00:00:00 2001 From: Tal Berlowitz <100570204+talber-nvidia@users.noreply.github.com> Date: Mon, 26 Sep 2022 19:30:38 +0300 Subject: [PATCH 28/51] Patch ifupdown2 (#9630) (#11548) --- ...alue-of-utils._execute_subprocess-me.patch | 21 +++++++++++++++++++ src/ifupdown2/patch/series | 1 + 2 files changed, 22 insertions(+) create mode 100644 src/ifupdown2/patch/0003-Fix-the-return-value-of-utils._execute_subprocess-me.patch diff --git a/src/ifupdown2/patch/0003-Fix-the-return-value-of-utils._execute_subprocess-me.patch b/src/ifupdown2/patch/0003-Fix-the-return-value-of-utils._execute_subprocess-me.patch new file mode 100644 index 000000000000..39cd481099ca --- /dev/null +++ b/src/ifupdown2/patch/0003-Fix-the-return-value-of-utils._execute_subprocess-me.patch @@ -0,0 +1,21 @@ +Fix the return value of utils._execute_subprocess method for empty strings +--- + ifupdown2/ifupdown/utils.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/ifupdown2/ifupdown/utils.py b/ifupdown2/ifupdown/utils.py +index d638fe9..0c5d8ce 100644 +--- a/ifupdown2/ifupdown/utils.py ++++ b/ifupdown2/ifupdown/utils.py +@@ -380,7 +380,7 @@ class utils(): + finally: + utils.disable_subprocess_signal_forwarding(signal.SIGINT) + +- cmd_output_string = cmd_output.decode() if cmd_output else cmd_output ++ cmd_output_string = cmd_output.decode() if cmd_output is not None else cmd_output + + if cmd_returncode != 0: + raise Exception(cls._format_error(cmd, +-- +2.14.1 + diff --git a/src/ifupdown2/patch/series b/src/ifupdown2/patch/series index c4e412bbe83f..7d0fa15ded61 100644 --- a/src/ifupdown2/patch/series +++ b/src/ifupdown2/patch/series @@ -1,2 +1,3 @@ 0001-fix-broadcast-addr-encoding.patch 0002-disable-checks-when-using-no-wait.patch +0003-Fix-the-return-value-of-utils._execute_subprocess-me.patch From 60c80ad26d17312b868a66ea32ccce3a1de2c8dc Mon Sep 17 00:00:00 2001 From: xumia <59720581+xumia@users.noreply.github.com> Date: Tue, 27 Sep 2022 06:55:19 +0800 Subject: [PATCH 29/51] [Build] Fix the build unstalbe issue caused by the kvm not ready (#12180) Why I did it Fix the build unstable issue caused by the kvm 9000 port is not ready to use in 2 seconds. 2022-09-02T10:57:30.8122304Z + /usr/bin/kvm -m 8192 -name onie -boot order=cd,once=d -cdrom target/files/bullseye/onie-recovery-x86_64-kvm_x86_64_4_asic-r0.iso -device e1000,netdev=onienet -netdev user,id=onienet,hostfwd=:0.0.0.0:3041-:22 -vnc 0.0.0.0:0 -vga std -drive file=target/sonic-6asic-vs.img,media=disk,if=virtio,index=0 -drive file=./sonic-installer.img,if=virtio,index=1 -serial telnet:127.0.0.1:9000,server 2022-09-02T10:57:30.8123378Z + sleep 2.0 2022-09-02T10:57:30.8123889Z + '[' -d /proc/284923 ']' 2022-09-02T10:57:30.8124528Z + echo 'to kill kvm: sudo kill 284923' 2022-09-02T10:57:30.8124994Z to kill kvm: sudo kill 284923 2022-09-02T10:57:30.8125362Z + ./install_sonic.py 2022-09-02T10:57:30.8125720Z Trying 127.0.0.1... 2022-09-02T10:57:30.8126041Z telnet: Unable to connect to remote host: Connection refused How I did it Waiting more time until the tcp port 9000 is ready, waiting for 60 seconds in maximum. --- scripts/build_kvm_image.sh | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/scripts/build_kvm_image.sh b/scripts/build_kvm_image.sh index 44009ed013f4..f3936a548299 100755 --- a/scripts/build_kvm_image.sh +++ b/scripts/build_kvm_image.sh @@ -49,6 +49,19 @@ prepare_installer_disk() umount $tmpdir } +wait_kvm_ready() +{ + local count=30 + local waiting_in_seconds=2.0 + for ((i=1; i<=$count; i++)); do + sleep $waiting_in_seconds + echo "$(date) [$i/$count] waiting for the port $KVM_PORT ready" + if netstat -l | grep -q ":$KVM_PORT"; then + break + fi + done +} + apt-get install -y net-tools create_disk prepare_installer_disk @@ -86,7 +99,7 @@ echo "Installing SONiC" kvm_pid=$! -sleep 2.0 +wait_kvm_ready [ -d "/proc/$kvm_pid" ] || { echo "ERROR: kvm died." @@ -114,7 +127,7 @@ echo "Booting up SONiC" kvm_pid=$! -sleep 2.0 +wait_kvm_ready [ -d "/proc/$kvm_pid" ] || { echo "ERROR: kvm died." From 1995540758cccbf4f3ff4c081a745db835993222 Mon Sep 17 00:00:00 2001 From: ShiyanWangMS Date: Tue, 27 Sep 2022 09:15:48 +0800 Subject: [PATCH 30/51] Upgrade docker-sonic-mgmt base image from Ubuntu18.04 to 20.04 (#12056) Upgrade docker-sonic-mgmt base image from Ubuntu18.04 to 20.04 --- dockers/docker-sonic-mgmt/Dockerfile.j2 | 27 ++++++++++++++++--------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/dockers/docker-sonic-mgmt/Dockerfile.j2 b/dockers/docker-sonic-mgmt/Dockerfile.j2 index fec662dff0ed..e803508e855f 100755 --- a/dockers/docker-sonic-mgmt/Dockerfile.j2 +++ b/dockers/docker-sonic-mgmt/Dockerfile.j2 @@ -1,5 +1,5 @@ {% set prefix = DEFAULT_CONTAINER_REGISTRY %} -FROM {{ prefix }}ubuntu:18.04 +FROM {{ prefix }}ubuntu:20.04 ENV DEBIAN_FRONTEND=noninteractive @@ -21,8 +21,6 @@ RUN apt-get update && apt-get install -y build-essential \ psmisc \ python \ python-dev \ - python-scapy \ - python-pip \ python3-pip \ python3-venv \ rsyslog \ @@ -31,10 +29,20 @@ RUN apt-get update && apt-get install -y build-essential \ sudo \ tcpdump \ telnet \ - vim + vim \ + python-is-python2 \ + software-properties-common + +RUN add-apt-repository -y universe +RUN curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py \ + && python2 get-pip.py + +RUN curl -L http://archive.ubuntu.com/ubuntu/pool/universe/s/scapy/python-scapy_2.3.3-3_all.deb \ + --output python-scapy_2.3.3-3_all.deb \ + && dpkg -i python-scapy_2.3.3-3_all.deb RUN pip install setuptools==44.1.1 -RUN pip install cffi==1.10.0 \ +RUN pip install cffi==1.12.0 \ contextlib2==0.6.0.post1 \ cryptography==3.3.2 \ "future>=0.16.0" \ @@ -96,7 +104,7 @@ RUN pip install cffi==1.10.0 \ && rm -f 1.0.0.tar.gz \ && pip install nnpy \ && pip install dpkt \ - && pip install scapy==2.4.5 --upgrade + && pip install scapy==2.4.5 --upgrade --ignore-installed # Install docker-ce-cli RUN apt-get update \ @@ -189,8 +197,7 @@ ENV PATH="$VIRTUAL_ENV/bin:$PATH" ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 PYTHONIOENCODING=UTF-8 -RUN python3 -m pip install --upgrade --ignore-installed pip setuptools==58.4.0 - +RUN python3 -m pip install --upgrade --ignore-installed pip setuptools==58.4.0 wheel==0.33.6 RUN python3 -m pip install setuptools-rust \ aiohttp \ defusedxml \ @@ -206,6 +213,7 @@ RUN python3 -m pip install setuptools-rust \ ixnetwork-restpy==1.0.64 \ ixnetwork-open-traffic-generator==0.0.79 \ snappi[ixnetwork,convergence]==0.7.44 \ + markupsafe==2.0.1 \ jinja2==2.7.2 \ jsonpatch \ lxml \ @@ -233,7 +241,6 @@ RUN python3 -m pip install setuptools-rust \ tabulate \ textfsm==1.1.2 \ virtualenv \ - wheel==0.33.6 \ pysubnettree \ nnpy \ dpkt \ @@ -248,5 +255,5 @@ RUN python3 -m pip install setuptools-rust \ celery[redis]==4.4.7 \ msrest==0.6.21 -# Deactivating a virtualenv. +# Deactivating a virtualenv ENV PATH="$BACKUP_OF_PATH" From 9c602320c3df3c8e84d5819c207b939b7ea84ff3 Mon Sep 17 00:00:00 2001 From: Ye Jianquan Date: Wed, 28 Sep 2022 11:38:41 +0800 Subject: [PATCH 31/51] install missed package python-dateutil (#12197) Why I did it Fix issue of can't import dateutil.parser in show_techsupport/test_auto_techsupport.py How I did it install python-dateutil --- dockers/docker-sonic-mgmt/Dockerfile.j2 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/dockers/docker-sonic-mgmt/Dockerfile.j2 b/dockers/docker-sonic-mgmt/Dockerfile.j2 index e803508e855f..7676341cbaed 100755 --- a/dockers/docker-sonic-mgmt/Dockerfile.j2 +++ b/dockers/docker-sonic-mgmt/Dockerfile.j2 @@ -86,6 +86,7 @@ RUN pip install cffi==1.12.0 \ allure-pytest==2.8.22 \ celery[redis]==4.4.7 \ msrest==0.6.21 \ + python-dateutil \ && git clone https://github.com/p4lang/scapy-vxlan.git \ && cd scapy-vxlan \ && python setup.py install \ @@ -253,7 +254,8 @@ RUN python3 -m pip install setuptools-rust \ ptf \ scapy==2.4.5 \ celery[redis]==4.4.7 \ - msrest==0.6.21 + msrest==0.6.21 \ + python-dateutil \ # Deactivating a virtualenv ENV PATH="$BACKUP_OF_PATH" From 7666af94030f44e48da1b4e8a8f265c32d0f2220 Mon Sep 17 00:00:00 2001 From: Ye Jianquan Date: Wed, 28 Sep 2022 14:39:33 +0800 Subject: [PATCH 32/51] Fix pip install error (#12198) Fix the error of pip install introduced in PR #12197 --- dockers/docker-sonic-mgmt/Dockerfile.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dockers/docker-sonic-mgmt/Dockerfile.j2 b/dockers/docker-sonic-mgmt/Dockerfile.j2 index 7676341cbaed..c6b4d1600a8e 100755 --- a/dockers/docker-sonic-mgmt/Dockerfile.j2 +++ b/dockers/docker-sonic-mgmt/Dockerfile.j2 @@ -255,7 +255,7 @@ RUN python3 -m pip install setuptools-rust \ scapy==2.4.5 \ celery[redis]==4.4.7 \ msrest==0.6.21 \ - python-dateutil \ + python-dateutil # Deactivating a virtualenv ENV PATH="$BACKUP_OF_PATH" From f890606d8252c0e181bf16ea0c83846d9a37bbbf Mon Sep 17 00:00:00 2001 From: Junchao-Mellanox <57339448+Junchao-Mellanox@users.noreply.github.com> Date: Wed, 28 Sep 2022 15:15:26 +0800 Subject: [PATCH 33/51] Revert "[Mellanox] Redirect ethtool stderr to subprocess for better error log (#12038)" (#12183) This reverts commit 9750cb4. There is a PR to handle 202205 branch revert: #12184 - Why I did it The PR to be reverted introduced many notice logs every 1 minute if SFP is not plugged: Cannot get module EEPROM information: Input/output error Before the "bad" PR, the message format is like this: INFO pmon#supervisord: xcvrd Cannot get module EEPROM information: Input/output error It was truncated by rsyslog because every message is the same. However, the "bad" PR introduces SFP index to the message: NOTICE pmon#xcvrd: Failed to get EEPROM data for sfp 39: Cannot get module EEPROM information: Input/output error Rsyslog no longer truncate such log and many such messages are flooded to syslog. - How I did it Revert the PR - How to verify it Manual test --- platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py b/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py index d373b6a8f705..617b4f33d636 100644 --- a/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py +++ b/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py @@ -359,8 +359,7 @@ def _read_eeprom_specific_bytes(self, offset, num_bytes): try: output = subprocess.check_output(ethtool_cmd, shell=True, - universal_newlines=True, - stderr=subprocess.PIPE) + universal_newlines=True) output_lines = output.splitlines() first_line_raw = output_lines[0] if "Offset" in first_line_raw: @@ -368,7 +367,6 @@ def _read_eeprom_specific_bytes(self, offset, num_bytes): line_split = line.split() eeprom_raw = eeprom_raw + line_split[1:] except subprocess.CalledProcessError as e: - logger.log_notice("Failed to get EEPROM data for sfp {}: {}".format(self.index, e.stderr)) return None eeprom_raw = list(map(lambda h: int(h, base=16), eeprom_raw)) From 4d317aff9491a56c094f96966bb0e3ff9d009611 Mon Sep 17 00:00:00 2001 From: Stephen Sun <5379172+stephenxs@users.noreply.github.com> Date: Wed, 28 Sep 2022 16:09:18 +0800 Subject: [PATCH 34/51] [Mellanox] Fix typo in platform API (#12136) - Why I did it Fix a typo in chassis platform API which causes the following error >>> import sonic_platform as P >>> c = P.platform.Platform().get_chassis() >>> sl = c.get_all_sfps() >>> sl[0].get_lpmode() Sep 28 07:48:33 INFO LOG: Initializing SX log with STDOUT as output file. False >>> del c Exception ignored in: Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/sonic_platform/chassis.py", line 126, in __del__ self.sfp_module.deinitialize_sdk_handle(sfp_module.SFP.shared_sdk_handle) NameError: name 'sfp_module' is not defined - How I did it Use self while using the SDK handle - How to verify it Manual test Signed-off-by: Stephen Sun --- platform/mellanox/mlnx-platform-api/sonic_platform/chassis.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform/mellanox/mlnx-platform-api/sonic_platform/chassis.py b/platform/mellanox/mlnx-platform-api/sonic_platform/chassis.py index b9fa2593174c..96a7b9e2315a 100644 --- a/platform/mellanox/mlnx-platform-api/sonic_platform/chassis.py +++ b/platform/mellanox/mlnx-platform-api/sonic_platform/chassis.py @@ -123,7 +123,7 @@ def __del__(self): if self._sfp_list: if self.sfp_module.SFP.shared_sdk_handle: - self.sfp_module.deinitialize_sdk_handle(sfp_module.SFP.shared_sdk_handle) + self.sfp_module.deinitialize_sdk_handle(self.sfp_module.SFP.shared_sdk_handle) @property def RJ45_port_list(self): From 8c7e0f8e02233dc4236f8f7f048b4281ab9d31b8 Mon Sep 17 00:00:00 2001 From: vijayvyasm <112528485+vijayvyasm@users.noreply.github.com> Date: Wed, 28 Sep 2022 18:37:33 -0700 Subject: [PATCH 35/51] Support for serdes platform library debian installation for Innovium SONiC image (#11920) Signed-off-by: vijayvyasm vijayvyasm@marvell.com Signed-off-by: vijayvyasm vijayvyasm@marvell.com --- platform/innovium/invm-sai.mk | 4 +++- platform/innovium/one-image.mk | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/platform/innovium/invm-sai.mk b/platform/innovium/invm-sai.mk index 25e490e44f83..734184e35b0c 100755 --- a/platform/innovium/invm-sai.mk +++ b/platform/innovium/invm-sai.mk @@ -5,11 +5,13 @@ INVM_SAI_ONLINE = https://github.com/Innovium/SONiC/raw/master/debian/master INVM_LIBSAI = isai.deb INVM_HSAI = saihdr.deb INVM_DRV = ipd.deb +INVM_SERDES_PLATFORM_LIBRARY = ivm_serdes_pltfm.deb $(INVM_LIBSAI)_URL = $(INVM_SAI_ONLINE)/$(INVM_LIBSAI) $(INVM_HSAI)_URL = $(INVM_SAI_ONLINE)/$(INVM_HSAI) $(INVM_DRV)_URL = $(INVM_SAI_ONLINE)/$(INVM_DRV) +$(INVM_SERDES_PLATFORM_LIBRARY)_URL = $(INVM_SAI_ONLINE)/$(INVM_SERDES_PLATFORM_LIBRARY) $(eval $(call add_conflict_package,$(INVM_HSAI),$(LIBSAIVS_DEV))) -SONIC_ONLINE_DEBS += $(INVM_LIBSAI) $(INVM_HSAI) $(INVM_DRV) +SONIC_ONLINE_DEBS += $(INVM_LIBSAI) $(INVM_HSAI) $(INVM_DRV) $(INVM_SERDES_PLATFORM_LIBRARY) diff --git a/platform/innovium/one-image.mk b/platform/innovium/one-image.mk index 2cae779d71ac..39e2e9fc8246 100755 --- a/platform/innovium/one-image.mk +++ b/platform/innovium/one-image.mk @@ -8,6 +8,6 @@ $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(CEL_MIDSTONE_200I_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(DELTA_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(NETBERG_AURORA_715_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(WISTRON_PLATFORM_MODULE) -$(SONIC_ONE_IMAGE)_INSTALLS += $(INVM_DRV) +$(SONIC_ONE_IMAGE)_INSTALLS += $(INVM_DRV) $(INVM_SERDES_PLATFORM_LIBRARY) $(SONIC_ONE_IMAGE)_DOCKERS += $(SONIC_INSTALL_DOCKER_IMAGES) SONIC_INSTALLERS += $(SONIC_ONE_IMAGE) From 1f9c89a8d34ab94cd871d0dd798d991d5e4cf00e Mon Sep 17 00:00:00 2001 From: Hua Liu <58683130+liuh-80@users.noreply.github.com> Date: Thu, 29 Sep 2022 10:27:57 +0800 Subject: [PATCH 36/51] [sonic-py-common] porting sonic_db_dump_load.py from sonic-py-swsssdk to sonic-py-common (#12185) Porting sonic_db_dump_load.py from sonic-py-swsssdk to sonic-py-common. #### Why I did it sonic-py-swsssdk will be deprecate, so porting sonic_db_dump_load.py to sonic-py-common. #### How I did it Copy sonic_db_dump_load.py to sonic-py-common, and fix minor API different. #### How to verify it Pass all E2E test. The platform_tests/test_advanced_reboot.py::test_warm_reboot will cover this script. #### Which release branch to backport (provide reason below if selected) - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 #### Description for the changelog Porting sonic_db_dump_load.py from sonic-py-swsssdk to sonic-py-common. #### Ensure to add label/tag for the feature raised. example - [PR#2174](https://github.com/sonic-net/sonic-utilities/pull/2174) where, Generic Config and Update feature has been labelled as GCU. #### Link to config_db schema for YANG module changes #### A picture of a cute animal (not mandatory but encouraged) --- src/sonic-py-common/setup.py | 6 + .../sonic_py_common/sonic_db_dump_load.py | 139 ++++++++++++++++++ 2 files changed, 145 insertions(+) create mode 100755 src/sonic-py-common/sonic_py_common/sonic_db_dump_load.py diff --git a/src/sonic-py-common/setup.py b/src/sonic-py-common/setup.py index f12c0d83cb77..144cf61f52c4 100644 --- a/src/sonic-py-common/setup.py +++ b/src/sonic-py-common/setup.py @@ -27,6 +27,12 @@ 'pytest', 'mock==3.0.5' # For python 2. Version >=4.0.0 drops support for py2 ], + entry_points={ + 'console_scripts': [ + 'sonic-db-load = sonic_py_common.sonic_db_dump_load:sonic_db_dump_load', + 'sonic-db-dump = sonic_py_common.sonic_db_dump_load:sonic_db_dump_load', + ], + }, classifiers=[ 'Intended Audience :: Developers', 'Operating System :: Linux', diff --git a/src/sonic-py-common/sonic_py_common/sonic_db_dump_load.py b/src/sonic-py-common/sonic_py_common/sonic_db_dump_load.py new file mode 100755 index 000000000000..126dd2bc112e --- /dev/null +++ b/src/sonic-py-common/sonic_py_common/sonic_db_dump_load.py @@ -0,0 +1,139 @@ +## ref: https://github.com/p/redis-dump-load/blob/7bbdb1eaea0a51ed4758d3ce6ca01d497a4e7429/redisdl.py + +def sonic_db_dump_load(): + import optparse + import os.path + import re + import sys + from redisdl import dump, load + from swsscommon.swsscommon import SonicDBConfig + + DUMP = 1 + LOAD = 2 + + def options_to_kwargs(options): + args = {} + if options.password: + args['password'] = options.password + if options.encoding: + args['encoding'] = options.encoding + # dump only + if hasattr(options, 'pretty') and options.pretty: + args['pretty'] = True + if hasattr(options, 'keys') and options.keys: + args['keys'] = options.keys + # load only + if hasattr(options, 'use_expireat') and options.use_expireat: + args['use_expireat'] = True + if hasattr(options, 'empty') and options.empty: + args['empty'] = True + if hasattr(options, 'backend') and options.backend: + args['streaming_backend'] = options.backend + if hasattr(options, 'dbname') and options.dbname: + if options.conntype == 'tcp': + args['host'] = SonicDBConfig.getDbHostname(options.dbname) + args['port'] = SonicDBConfig.getDbPort(options.dbname) + args['db'] = SonicDBConfig.getDbId(options.dbname) + args['unix_socket_path'] = None + elif options.conntype == "unix_socket": + args['host'] = None + args['port'] = None + args['db'] = SonicDBConfig.getDbId(options.dbname) + args['unix_socket_path'] = SonicDBConfig.getDbSock(options.dbname) + else: + raise TypeError('redis connection type is tcp or unix_socket') + + return args + + def do_dump(options): + if options.output: + output = open(options.output, 'w') + else: + output = sys.stdout + + kwargs = options_to_kwargs(options) + dump(output, **kwargs) + + if options.output: + output.close() + + def do_load(options, args): + if len(args) > 0: + input = open(args[0], 'rb') + else: + input = sys.stdin + + kwargs = options_to_kwargs(options) + load(input, **kwargs) + + if len(args) > 0: + input.close() + + script_name = os.path.basename(sys.argv[0]) + if re.search(r'load(?:$|\.)', script_name): + action = help = LOAD + elif re.search(r'dump(?:$|\.)', script_name): + action = help = DUMP + else: + # default is dump, however if dump is specifically requested + # we don't show help text for toggling between dumping and loading + action = DUMP + help = None + + if help == LOAD: + usage = "Usage: %prog [options] [FILE]" + usage += "\n\nLoad data from FILE (which must be a JSON dump previously created" + usage += "\nby redisdl) into specified or default redis." + usage += "\n\nIf FILE is omitted standard input is read." + elif help == DUMP: + usage = "Usage: %prog [options]" + usage += "\n\nDump data from specified or default redis." + usage += "\n\nIf no output file is specified, dump to standard output." + else: + usage = "Usage: %prog [options]" + usage += "\n %prog -l [options] [FILE]" + usage += "\n\nDump data from redis or load data into redis." + usage += "\n\nIf input or output file is specified, dump to standard output and load" + usage += "\nfrom standard input." + parser = optparse.OptionParser(usage=usage) + parser.add_option('-w', '--password', help='connect with PASSWORD') + if help == DUMP: + parser.add_option('-n', '--dbname', help='dump DATABASE (APPL_DB/ASIC_DB...)') + parser.add_option('-t', '--conntype', help='indicate redis connection type (tcp[default] or unix_socket)', default='tcp') + parser.add_option('-k', '--keys', help='dump only keys matching specified glob-style pattern') + parser.add_option('-o', '--output', help='write to OUTPUT instead of stdout') + parser.add_option('-y', '--pretty', help='split output on multiple lines and indent it', action='store_true') + parser.add_option('-E', '--encoding', help='set encoding to use while decoding data from redis', default='utf-8') + elif help == LOAD: + parser.add_option('-n', '--dbname', help='dump DATABASE (APPL_DB/ASIC_DB...)') + parser.add_option('-t', '--conntype', help='indicate redis connection type (tcp[default] or unix_socket)', default='tcp') + parser.add_option('-e', '--empty', help='delete all keys in destination db prior to loading', action='store_true') + parser.add_option('-E', '--encoding', help='set encoding to use while encoding data to redis', default='utf-8') + parser.add_option('-B', '--backend', help='use specified streaming backend') + parser.add_option('-A', '--use-expireat', help='use EXPIREAT rather than TTL/EXPIRE', action='store_true') + else: + parser.add_option('-l', '--load', help='load data into redis (default is to dump data from redis)', action='store_true') + parser.add_option('-n', '--dbname', help='dump DATABASE (APPL_DB/ASIC_DB/COUNTERS_DB/LOGLEVEL_DB/CONFIG_DB...)') + parser.add_option('-t', '--conntype', help='indicate redis connection type (tcp[default] or unix_socket)', default='tcp') + parser.add_option('-k', '--keys', help='dump only keys matching specified glob-style pattern') + parser.add_option('-o', '--output', help='write to OUTPUT instead of stdout (dump mode only)') + parser.add_option('-y', '--pretty', help='split output on multiple lines and indent it (dump mode only)', action='store_true') + parser.add_option('-e', '--empty', help='delete all keys in destination db prior to loading (load mode only)', action='store_true') + parser.add_option('-E', '--encoding', help='set encoding to use while decoding data from redis', default='utf-8') + parser.add_option('-A', '--use-expireat', help='use EXPIREAT rather than TTL/EXPIRE', action='store_true') + parser.add_option('-B', '--backend', help='use specified streaming backend (load mode only)') + options, args = parser.parse_args() + + if hasattr(options, 'load') and options.load: + action = LOAD + + if action == DUMP: + if len(args) > 0: + parser.print_help() + exit(4) + do_dump(options) + else: + if len(args) > 1: + parser.print_help() + exit(4) + do_load(options, args) From d9c9c70fb543ed4e6c8bfff24e17e65613409e72 Mon Sep 17 00:00:00 2001 From: Dmytro Lytvynenko Date: Fri, 30 Sep 2022 01:12:01 +0300 Subject: [PATCH 37/51] [BFN] Move qsfp eeprom reading to new cached api (#9909) * Move qsfp eeprom reading to new cached api * provide reading multiple pages in recursive manner * workaround with flat memory on cmis * remove workaround with memory model * Remove unused imports --- .../sonic_platform/platform_thrift_client.py | 11 ++-- .../sonic_platform/pltfm_mgr_rpc/ttypes.py | 18 ------ .../sonic_platform/sfp.py | 62 +++++++++++++------ 3 files changed, 48 insertions(+), 43 deletions(-) diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_thrift_client.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_thrift_client.py index dff16577de74..8490d132a2df 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_thrift_client.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_thrift_client.py @@ -1,12 +1,10 @@ #!/usr/bin/env python try: - import os - import sys import time - import importlib - sys.path.append(os.path.dirname(__file__)) + from sonic_platform.pltfm_mgr_rpc.pltfm_mgr_rpc import Client + from sonic_platform.pltfm_mgr_rpc.pltfm_mgr_rpc import InvalidPltfmMgrOperation from thrift.transport import TSocket from thrift.transport import TTransport @@ -25,9 +23,8 @@ def open(self): self.transport = TTransport.TBufferedTransport(self.transport) bprotocol = TBinaryProtocol.TBinaryProtocol(self.transport) - self.pltfm_mgr_module = importlib.import_module(".".join(["pltfm_mgr_rpc", "pltfm_mgr_rpc"])) pltfm_mgr_protocol = TMultiplexedProtocol.TMultiplexedProtocol(bprotocol, "pltfm_mgr_rpc") - self.pltfm_mgr = self.pltfm_mgr_module.Client(pltfm_mgr_protocol) + self.pltfm_mgr = Client(pltfm_mgr_protocol) self.transport.open() return self @@ -59,7 +56,7 @@ def pltfm_mgr_try(func, default=None, thrift_attempts=35): def pm_cb_run(client): try: return (None, func(client.pltfm_mgr)) - except client.pltfm_mgr_module.InvalidPltfmMgrOperation as ouch: + except InvalidPltfmMgrOperation as ouch: return (ouch.code, default) return thrift_try(pm_cb_run) diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py index 501596941664..391d4bd5377b 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py @@ -16,24 +16,6 @@ all_structs = [] -class qsfp_eeprom_page_t(object): - PAGE0_LOWER = 0 - PAGE0_UPPER = 1 - PAGE3 = 2 - - _VALUES_TO_NAMES = { - 0: "PAGE0_LOWER", - 1: "PAGE0_UPPER", - 2: "PAGE3", - } - - _NAMES_TO_VALUES = { - "PAGE0_LOWER": 0, - "PAGE0_UPPER": 1, - "PAGE3": 2, - } - - class pltfm_mgr_sys_tmp_t(object): """ Attributes: diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/sfp.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/sfp.py index f5d800b749c9..6a5534d8ba6b 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/sfp.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/sfp.py @@ -11,29 +11,38 @@ SFP_TYPE = "SFP" QSFP_TYPE = "QSFP" QSFP_DD_TYPE = "QSFP_DD" +EEPROM_PAGE_SIZE = 128 +try: + from thrift.Thrift import TApplicationException + + def cached_num_bytes_get(client): + return client.pltfm_mgr.pltfm_mgr_qsfp_cached_num_bytes_get(1, 0, 0, 0) + thrift_try(cached_num_bytes_get, 1) + EEPROM_CACHED_API_SUPPORT = True +except TApplicationException as e: + EEPROM_CACHED_API_SUPPORT = False class Sfp(SfpOptoeBase): """ BFN Platform-specific SFP class """ - SFP_EEPROM_PATH = "/var/run/platform/sfp/" - def __init__(self, port_num): SfpOptoeBase.__init__(self) self.index = port_num self.port_num = port_num self.sfp_type = QSFP_TYPE + self.SFP_EEPROM_PATH = "/var/run/platform/sfp/" - if not os.path.exists(self.SFP_EEPROM_PATH): - try: - os.makedirs(self.SFP_EEPROM_PATH) - except OSError as e: - if e.errno != errno.EEXIST: - raise - - self.eeprom_path = self.SFP_EEPROM_PATH + "sfp{}-eeprom-cache".format(self.index) + if not EEPROM_CACHED_API_SUPPORT: + if not os.path.exists(self.SFP_EEPROM_PATH): + try: + os.makedirs(self.SFP_EEPROM_PATH) + except OSError as e: + if e.errno != errno.EEXIST: + raise + self.eeprom_path = self.SFP_EEPROM_PATH + "sfp{}-eeprom-cache".format(self.index) def get_presence(self): """ @@ -47,7 +56,7 @@ def qsfp_presence_get(client): try: presence = thrift_try(qsfp_presence_get) except Exception as e: - print( e.__doc__) + print(e.__doc__) print(e.message) return presence @@ -75,14 +84,31 @@ def get_eeprom_path(self): def qsfp_info_get(client): return client.pltfm_mgr.pltfm_mgr_qsfp_info_get(self.index) - if self.get_presence(): - eeprom_hex = thrift_try(qsfp_info_get) - eeprom_raw = bytearray.fromhex(eeprom_hex) - with open(self.eeprom_path, 'wb') as fp: - fp.write(eeprom_raw) - return self.eeprom_path + eeprom_hex = thrift_try(qsfp_info_get) + eeprom_raw = bytearray.fromhex(eeprom_hex) + with open(self.eeprom_path, 'wb') as fp: + fp.write(eeprom_raw) + return self.eeprom_path + + def read_eeprom(self, offset, num_bytes): + if not self.get_presence(): + return None + + if not EEPROM_CACHED_API_SUPPORT: + return super().read_eeprom(offset, num_bytes) + + def cached_num_bytes_get(page, offset, num_bytes): + def qsfp_cached_num_bytes_get(client): + return client.pltfm_mgr.pltfm_mgr_qsfp_cached_num_bytes_get(self.index, page, offset, num_bytes) + return bytearray.fromhex(thrift_try(qsfp_cached_num_bytes_get)) + + page_offset = offset % EEPROM_PAGE_SIZE + if page_offset + num_bytes > EEPROM_PAGE_SIZE: + curr_page_num_bytes_left = EEPROM_PAGE_SIZE - page_offset + curr_page_bytes = cached_num_bytes_get(offset // EEPROM_PAGE_SIZE, page_offset, curr_page_num_bytes_left) + return curr_page_bytes + self.read_eeprom(offset + curr_page_num_bytes_left, num_bytes - curr_page_num_bytes_left) - return None + return cached_num_bytes_get(offset // EEPROM_PAGE_SIZE, page_offset, num_bytes) def write_eeprom(self, offset, num_bytes, write_buffer): # Not supported at the moment From d08fcc971cbd7b27c7140eaa7b1baaad433ddba8 Mon Sep 17 00:00:00 2001 From: Dmytro Lytvynenko Date: Fri, 30 Sep 2022 01:13:46 +0300 Subject: [PATCH 38/51] [BFN] Updated syseeprom platform plugin to use onie-eeprom (#10556) * Align system eeprom info with ONIE * revert linked sonic_platform implementation * refactor into one class * refactor after review --- .../sonic_platform/chassis.py | 32 ++-- .../sonic_platform/eeprom.py | 155 +++++++++------- .../pltfm_mgr_rpc/pltfm_mgr_rpc.py | 175 ++++++++++++++++++ .../sonic_platform/pltfm_mgr_rpc/ttypes.py | 62 +++++++ 4 files changed, 336 insertions(+), 88 deletions(-) diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/chassis.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/chassis.py index 6d0e8b8c120c..1041561db423 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/chassis.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/chassis.py @@ -13,8 +13,8 @@ from sonic_platform.psu import psu_list_get from sonic_platform.fan_drawer import fan_drawer_list_get from sonic_platform.thermal import thermal_list_get - from eeprom import Eeprom - from platform_utils import file_create + from sonic_platform.platform_utils import file_create + from sonic_platform.eeprom import Eeprom from sonic_platform.platform_thrift_client import pltfm_mgr_ready from sonic_platform.platform_thrift_client import thrift_try @@ -40,7 +40,10 @@ class Chassis(ChassisBase): def __init__(self): ChassisBase.__init__(self) - self.__eeprom = None + self._eeprom = Eeprom() + self.__tlv_bin_eeprom = self._eeprom.get_raw_data() + self.__tlv_dict_eeprom = self._eeprom.get_data() + self.__fan_drawers = None self.__fan_list = None self.__thermals = None @@ -57,16 +60,6 @@ def __init__(self): file_create(config_dict['handlers']['file']['filename'], '646') logging.config.dictConfig(config_dict) - @property - def _eeprom(self): - if self.__eeprom is None: - self.__eeprom = Eeprom() - return self.__eeprom - - @_eeprom.setter - def _eeprom(self, value): - pass - @property def _fan_drawer_list(self): if self.__fan_drawers is None: @@ -152,7 +145,7 @@ def get_name(self): Returns: string: The name of the chassis """ - return self._eeprom.modelstr() + return self._eeprom.modelstr(self.__tlv_bin_eeprom) def get_presence(self): """ @@ -168,7 +161,7 @@ def get_model(self): Returns: string: Model/part number of chassis """ - return self._eeprom.part_number_str() + return self._eeprom.part_number_str(self.__tlv_bin_eeprom) def get_serial(self): """ @@ -176,7 +169,7 @@ def get_serial(self): Returns: string: Serial number of chassis """ - return self._eeprom.serial_number_str() + return self._eeprom.serial_number_str(self.__tlv_bin_eeprom) def get_revision(self): """ @@ -184,7 +177,8 @@ def get_revision(self): Returns: string: Revision number of chassis """ - return self._eeprom.revision_str() + return self.__tlv_dict_eeprom.get( + "0x{:X}".format(Eeprom._TLV_CODE_LABEL_REVISION), 'N/A') def get_sfp(self, index): """ @@ -225,7 +219,7 @@ def get_base_mac(self): A string containing the MAC address in the format 'XX:XX:XX:XX:XX:XX' """ - return self._eeprom.base_mac_addr() + return self._eeprom.base_mac_addr(self.__tlv_bin_eeprom) def get_system_eeprom_info(self): """ @@ -236,7 +230,7 @@ def get_system_eeprom_info(self): OCP ONIE TlvInfo EEPROM format and values are their corresponding values. """ - return self._eeprom.system_eeprom_info() + return self.__tlv_dict_eeprom def __get_transceiver_change_event(self, timeout=0): forever = False diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/eeprom.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/eeprom.py index 2335c02863d9..4b5c1e3051fb 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/eeprom.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/eeprom.py @@ -2,7 +2,8 @@ import os import sys import datetime - import re + import logging + import logging.config sys.path.append(os.path.dirname(__file__)) @@ -13,13 +14,15 @@ from sonic_platform_base.sonic_eeprom import eeprom_base from sonic_platform_base.sonic_eeprom import eeprom_tlvinfo - from platform_utils import file_create - from platform_thrift_client import thrift_try + from sonic_py_common import device_info + + from sonic_platform.platform_thrift_client import thrift_try + from sonic_platform.platform_utils import file_create + except ImportError as e: raise ImportError (str(e) + "- required module not found") - _platform_eeprom_map = { "prod_name" : ("Product Name", "0x21", 12), "odm_pcba_part_num" : ("Part Number", "0x22", 13), @@ -44,25 +47,55 @@ class Eeprom(eeprom_tlvinfo.TlvInfoDecoder): def __init__(self): file_create(_EEPROM_SYMLINK, '646') file_create(_EEPROM_STATUS, '646') - with open(_EEPROM_STATUS, 'w') as f: - f.write("initializing..") + super(Eeprom, self).__init__(_EEPROM_SYMLINK, 0, _EEPROM_STATUS, True) - self.eeprom_path = _EEPROM_SYMLINK - super(Eeprom, self).__init__(self.eeprom_path, 0, _EEPROM_STATUS, True) - - def sys_eeprom_get(client): - return client.pltfm_mgr.pltfm_mgr_sys_eeprom_get() + self._eeprom_bin = bytearray() + self.report_status("initializing..") try: - platform_eeprom = thrift_try(sys_eeprom_get) - except Exception: - raise RuntimeError("eeprom.py: Initialization failed") + try: + if device_info.get_platform() in ["x86_64-accton_as9516_32d-r0", + "x86_64-accton_as9516bf_32d-r0"]: + def tlv_eeprom_get(client): + return client.pltfm_mgr.pltfm_mgr_tlv_eeprom_get() + try: + self._eeprom_bin = bytearray.fromhex( + thrift_try(tlv_eeprom_get, 1).raw_content_hex) + except TApplicationException as e: + raise RuntimeError("api is not supported") + except Exception as e: + self._eeprom_bin = bytearray.fromhex( + thrift_try(tlv_eeprom_get).raw_content_hex) + else: + raise RuntimeError("platform is not supported") + + except RuntimeError as e: + logging.warning("Tlv eeprom fetching failed: %s, using OpenBMC" % (str(e))) + + def sys_eeprom_get(client): + return client.pltfm_mgr.pltfm_mgr_sys_eeprom_get() + + eeprom_params = self.platfrom_eeprom_to_params(thrift_try(sys_eeprom_get)) + stdout_stream = sys.stdout + sys.stdout = open(os.devnull, 'w') + self._eeprom_bin = self.set_eeprom(self._eeprom_bin, [eeprom_params]) + sys.stdout.close() + sys.stdout = stdout_stream + try: + self.write_eeprom(self._eeprom_bin) + self.report_status("ok") + except IOError as e: + logging.error("Failed to write eeprom: %s" % (str(e))) - self.__eeprom_init(platform_eeprom) + except Exception as e: + logging.error("eeprom.py: Initialization failed: %s" % (str(e))) + raise RuntimeError("eeprom.py: Initialization failed: %s" % (str(e))) - def __eeprom_init(self, platform_eeprom): - with open(_EEPROM_STATUS, 'w') as f: - f.write("ok") + self._system_eeprom_info = dict() + visitor = EepromContentVisitor(self._system_eeprom_info) + self.visit_eeprom(self._eeprom_bin, visitor) + @staticmethod + def platfrom_eeprom_to_params(platform_eeprom): eeprom_params = "" for attr, val in platform_eeprom.__dict__.items(): if val is None: @@ -86,57 +119,41 @@ def __eeprom_init(self, platform_eeprom): if len(eeprom_params) > 0: eeprom_params += "," eeprom_params += "{0:s}={1:s}".format(elem[1], value) + return eeprom_params - orig_stdout = sys.stdout - sys.stdout = StringIO() - try: - eeprom_data = eeprom_tlvinfo.TlvInfoDecoder.set_eeprom(self, "", [eeprom_params]) - finally: - decode_output = sys.stdout.getvalue() - sys.stdout = orig_stdout - - eeprom_base.EepromDecoder.write_eeprom(self, eeprom_data) - self.__eeprom_tlv_dict = self.__parse_output(decode_output) + def get_data(self): + return self._system_eeprom_info - def __parse_output(self, decode_output): - EEPROM_DECODE_HEADLINES = 6 - lines = decode_output.replace('\0', '').split('\n') - lines = lines[EEPROM_DECODE_HEADLINES:] - res = dict() - - for line in lines: - try: - # match whitespace-separated tag hex, length and value (value is mathced with its whitespaces) - match = re.search('(0x[0-9a-fA-F]{2})([\s]+[\S]+[\s]+)([\S]+[\s]*[\S]*)', line) - if match is not None: - code = match.group(1) - value = match.group(3).rstrip('\0') - res[code] = value - except Exception: - pass - return res - - def __tlv_get(self, code): - return self.__eeprom_tlv_dict.get("0x{:X}".format(code), 'N/A') - - def system_eeprom_info(self): - return self.__eeprom_tlv_dict - - def serial_number_str(self): - return self.__tlv_get(self._TLV_CODE_SERIAL_NUMBER) - - def serial_str(self): - return self.serial_number_str() - - def base_mac_addr(self): - return self.__tlv_get(self._TLV_CODE_MAC_BASE) - - def part_number_str(self): - return self.__tlv_get(self._TLV_CODE_PART_NUMBER) - - def modelstr(self): - return self.__tlv_get(self._TLV_CODE_PRODUCT_NAME) - - def revision_str(self): - return self.__tlv_get(self._TLV_CODE_LABEL_REVISION) + def get_raw_data(self): + return self._eeprom_bin + def report_status(self, status): + status_file = None + try: + status_file = open(_EEPROM_STATUS, "w") + status_file.write(status) + except IOError as e: + logging.error("Failed to report state: %s" % (str(e))) + finally: + if status_file is not None: + status_file.close() + +class EepromContentVisitor(eeprom_tlvinfo.EepromDefaultVisitor): + def __init__(self, content_dict): + self.content_dict = content_dict + + def visit_tlv(self, name, code, length, value): + if code != Eeprom._TLV_CODE_VENDOR_EXT: + self.content_dict["0x{:X}".format(code)] = value.rstrip('\0') + else: + if value: + value = value.rstrip('\0') + if value: + code = "0x{:X}".format(code) + if code not in self.content_dict: + self.content_dict[code] = [value] + else: + self.content_dict[code].append(value) + + def set_error(self, error): + logging.error("EepromContentVisitor error: %s" % (str(error))) diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/pltfm_mgr_rpc.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/pltfm_mgr_rpc.py index 0fa03d58b31a..b671be1be313 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/pltfm_mgr_rpc.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/pltfm_mgr_rpc.py @@ -33,6 +33,9 @@ def pltfm_mgr_sys_tmp_get(self): def pltfm_mgr_sys_eeprom_get(self): pass + def pltfm_mgr_tlv_eeprom_get(self): + pass + def pltfm_mgr_pwr_supply_present_get(self, ps_num): """ Parameters: @@ -403,6 +406,34 @@ def recv_pltfm_mgr_sys_eeprom_get(self): raise result.ouch raise TApplicationException(TApplicationException.MISSING_RESULT, "pltfm_mgr_sys_eeprom_get failed: unknown result") + def pltfm_mgr_tlv_eeprom_get(self): + self.send_pltfm_mgr_tlv_eeprom_get() + return self.recv_pltfm_mgr_tlv_eeprom_get() + + def send_pltfm_mgr_tlv_eeprom_get(self): + self._oprot.writeMessageBegin('pltfm_mgr_tlv_eeprom_get', TMessageType.CALL, self._seqid) + args = pltfm_mgr_tlv_eeprom_get_args() + args.write(self._oprot) + self._oprot.writeMessageEnd() + self._oprot.trans.flush() + + def recv_pltfm_mgr_tlv_eeprom_get(self): + iprot = self._iprot + (fname, mtype, rseqid) = iprot.readMessageBegin() + if mtype == TMessageType.EXCEPTION: + x = TApplicationException() + x.read(iprot) + iprot.readMessageEnd() + raise x + result = pltfm_mgr_tlv_eeprom_get_result() + result.read(iprot) + iprot.readMessageEnd() + if result.success is not None: + return result.success + if result.ouch is not None: + raise result.ouch + raise TApplicationException(TApplicationException.MISSING_RESULT, "pltfm_mgr_tlv_eeprom_get failed: unknown result") + def pltfm_mgr_pwr_supply_present_get(self, ps_num): """ Parameters: @@ -1579,6 +1610,7 @@ def __init__(self, handler): self._processMap["pltfm_mgr_dummy"] = Processor.process_pltfm_mgr_dummy self._processMap["pltfm_mgr_sys_tmp_get"] = Processor.process_pltfm_mgr_sys_tmp_get self._processMap["pltfm_mgr_sys_eeprom_get"] = Processor.process_pltfm_mgr_sys_eeprom_get + self._processMap["pltfm_mgr_tlv_eeprom_get"] = Processor.process_pltfm_mgr_tlv_eeprom_get self._processMap["pltfm_mgr_pwr_supply_present_get"] = Processor.process_pltfm_mgr_pwr_supply_present_get self._processMap["pltfm_mgr_pwr_supply_info_get"] = Processor.process_pltfm_mgr_pwr_supply_info_get self._processMap["pltfm_mgr_pwr_rail_info_get"] = Processor.process_pltfm_mgr_pwr_rail_info_get @@ -1710,6 +1742,32 @@ def process_pltfm_mgr_sys_eeprom_get(self, seqid, iprot, oprot): oprot.writeMessageEnd() oprot.trans.flush() + def process_pltfm_mgr_tlv_eeprom_get(self, seqid, iprot, oprot): + args = pltfm_mgr_tlv_eeprom_get_args() + args.read(iprot) + iprot.readMessageEnd() + result = pltfm_mgr_tlv_eeprom_get_result() + try: + result.success = self._handler.pltfm_mgr_tlv_eeprom_get() + msg_type = TMessageType.REPLY + except TTransport.TTransportException: + raise + except InvalidPltfmMgrOperation as ouch: + msg_type = TMessageType.REPLY + result.ouch = ouch + except TApplicationException as ex: + logging.exception('TApplication exception in handler') + msg_type = TMessageType.EXCEPTION + result = ex + except Exception: + logging.exception('Unexpected exception in handler') + msg_type = TMessageType.EXCEPTION + result = TApplicationException(TApplicationException.INTERNAL_ERROR, 'Internal error') + oprot.writeMessageBegin("pltfm_mgr_tlv_eeprom_get", msg_type, seqid) + result.write(oprot) + oprot.writeMessageEnd() + oprot.trans.flush() + def process_pltfm_mgr_pwr_supply_present_get(self, seqid, iprot, oprot): args = pltfm_mgr_pwr_supply_present_get_args() args.read(iprot) @@ -2954,6 +3012,123 @@ def __ne__(self, other): ) +class pltfm_mgr_tlv_eeprom_get_args(object): + + + def read(self, iprot): + if iprot._fast_decode is not None and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None: + iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec]) + return + iprot.readStructBegin() + while True: + (fname, ftype, fid) = iprot.readFieldBegin() + if ftype == TType.STOP: + break + else: + iprot.skip(ftype) + iprot.readFieldEnd() + iprot.readStructEnd() + + def write(self, oprot): + if oprot._fast_encode is not None and self.thrift_spec is not None: + oprot.trans.write(oprot._fast_encode(self, [self.__class__, self.thrift_spec])) + return + oprot.writeStructBegin('pltfm_mgr_tlv_eeprom_get_args') + oprot.writeFieldStop() + oprot.writeStructEnd() + + def validate(self): + return + + def __repr__(self): + L = ['%s=%r' % (key, value) + for key, value in self.__dict__.items()] + return '%s(%s)' % (self.__class__.__name__, ', '.join(L)) + + def __eq__(self, other): + return isinstance(other, self.__class__) and self.__dict__ == other.__dict__ + + def __ne__(self, other): + return not (self == other) +all_structs.append(pltfm_mgr_tlv_eeprom_get_args) +pltfm_mgr_tlv_eeprom_get_args.thrift_spec = ( +) + + +class pltfm_mgr_tlv_eeprom_get_result(object): + """ + Attributes: + - success + - ouch + + """ + + + def __init__(self, success=None, ouch=None,): + self.success = success + self.ouch = ouch + + def read(self, iprot): + if iprot._fast_decode is not None and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None: + iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec]) + return + iprot.readStructBegin() + while True: + (fname, ftype, fid) = iprot.readFieldBegin() + if ftype == TType.STOP: + break + if fid == 0: + if ftype == TType.STRUCT: + self.success = pltfm_mgr_tlv_sys_eeprom_t() + self.success.read(iprot) + else: + iprot.skip(ftype) + elif fid == 1: + if ftype == TType.STRUCT: + self.ouch = InvalidPltfmMgrOperation.read(iprot) + else: + iprot.skip(ftype) + else: + iprot.skip(ftype) + iprot.readFieldEnd() + iprot.readStructEnd() + + def write(self, oprot): + if oprot._fast_encode is not None and self.thrift_spec is not None: + oprot.trans.write(oprot._fast_encode(self, [self.__class__, self.thrift_spec])) + return + oprot.writeStructBegin('pltfm_mgr_tlv_eeprom_get_result') + if self.success is not None: + oprot.writeFieldBegin('success', TType.STRUCT, 0) + self.success.write(oprot) + oprot.writeFieldEnd() + if self.ouch is not None: + oprot.writeFieldBegin('ouch', TType.STRUCT, 1) + self.ouch.write(oprot) + oprot.writeFieldEnd() + oprot.writeFieldStop() + oprot.writeStructEnd() + + def validate(self): + return + + def __repr__(self): + L = ['%s=%r' % (key, value) + for key, value in self.__dict__.items()] + return '%s(%s)' % (self.__class__.__name__, ', '.join(L)) + + def __eq__(self, other): + return isinstance(other, self.__class__) and self.__dict__ == other.__dict__ + + def __ne__(self, other): + return not (self == other) +all_structs.append(pltfm_mgr_tlv_eeprom_get_result) +pltfm_mgr_tlv_eeprom_get_result.thrift_spec = ( + (0, TType.STRUCT, 'success', [pltfm_mgr_tlv_sys_eeprom_t, None], None, ), # 0 + (1, TType.STRUCT, 'ouch', [InvalidPltfmMgrOperation, None], None, ), # 1 +) + + class pltfm_mgr_pwr_supply_present_get_args(object): """ Attributes: diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py index 391d4bd5377b..ad686b888029 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/pltfm_mgr_rpc/ttypes.py @@ -460,6 +460,63 @@ def __ne__(self, other): return not (self == other) +class pltfm_mgr_tlv_sys_eeprom_t(object): + """ + Attributes: + - raw_content_hex + + """ + + + def __init__(self, raw_content_hex=None,): + self.raw_content_hex = raw_content_hex + + def read(self, iprot): + if iprot._fast_decode is not None and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None: + iprot._fast_decode(self, iprot, [self.__class__, self.thrift_spec]) + return + iprot.readStructBegin() + while True: + (fname, ftype, fid) = iprot.readFieldBegin() + if ftype == TType.STOP: + break + if fid == 1: + if ftype == TType.STRING: + self.raw_content_hex = iprot.readString().decode('utf-8', errors='replace') if sys.version_info[0] == 2 else iprot.readString() + else: + iprot.skip(ftype) + else: + iprot.skip(ftype) + iprot.readFieldEnd() + iprot.readStructEnd() + + def write(self, oprot): + if oprot._fast_encode is not None and self.thrift_spec is not None: + oprot.trans.write(oprot._fast_encode(self, [self.__class__, self.thrift_spec])) + return + oprot.writeStructBegin('pltfm_mgr_tlv_sys_eeprom_t') + if self.raw_content_hex is not None: + oprot.writeFieldBegin('raw_content_hex', TType.STRING, 1) + oprot.writeString(self.raw_content_hex.encode('utf-8') if sys.version_info[0] == 2 else self.raw_content_hex) + oprot.writeFieldEnd() + oprot.writeFieldStop() + oprot.writeStructEnd() + + def validate(self): + return + + def __repr__(self): + L = ['%s=%r' % (key, value) + for key, value in self.__dict__.items()] + return '%s(%s)' % (self.__class__.__name__, ', '.join(L)) + + def __eq__(self, other): + return isinstance(other, self.__class__) and self.__dict__ == other.__dict__ + + def __ne__(self, other): + return not (self == other) + + class pltfm_mgr_pwr_supply_info_t(object): """ Attributes: @@ -1380,6 +1437,11 @@ def __ne__(self, other): (21, TType.STRING, 'location', 'UTF8', None, ), # 21 (22, TType.I16, 'crc8', None, None, ), # 22 ) +all_structs.append(pltfm_mgr_tlv_sys_eeprom_t) +pltfm_mgr_tlv_sys_eeprom_t.thrift_spec = ( + None, # 0 + (1, TType.STRING, 'raw_content_hex', 'UTF8', None, ), # 1 +) all_structs.append(pltfm_mgr_pwr_supply_info_t) pltfm_mgr_pwr_supply_info_t.thrift_spec = ( None, # 0 From 9bb0a7f33cd99f769063dd4bb798f0b59985e619 Mon Sep 17 00:00:00 2001 From: Andriy Kokhan Date: Fri, 30 Sep 2022 01:18:43 +0300 Subject: [PATCH 39/51] [BFN] Canceling PSU platform API calls on SIGTERM (#10720) * [BFN] Canceling PSU platform API calls on SIGTERM Signed-off-by: Andriy Kokhan * [BFN] Fixed SONiC fwutil exec time (#31) Signed-off-by: Taras Keryk Signed-off-by: Andriy Kokhan Signed-off-by: Taras Keryk Co-authored-by: Taras Keryk --- .../sonic_platform/component.py | 2 + .../sonic_platform/platform_utils.py | 60 ++++++++++++++++++- .../sonic_platform/psu.py | 30 +++++++++- 3 files changed, 89 insertions(+), 3 deletions(-) diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/component.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/component.py index 47a0993bf3e5..a7f236cb42a4 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/component.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/component.py @@ -6,6 +6,7 @@ import json from collections import OrderedDict from sonic_py_common import device_info + from platform_utils import limit_execution_time except ImportError as e: raise ImportError(str(e) + "- required module not found") @@ -24,6 +25,7 @@ def get_bios_version(): except subprocess.CalledProcessError as e: raise RuntimeError("Failed to get BIOS version") +@limit_execution_time(1) def get_bmc_version(): """ Retrieves the firmware version of the BMC diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_utils.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_utils.py index 81e78ee01041..2f7b5aecb6d0 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_utils.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/platform_utils.py @@ -3,11 +3,19 @@ try: import os import subprocess + import signal + from functools import wraps except ImportError as e: raise ImportError(str(e) + "- required module not found") def file_create(path, mode=None): + """ + Ensure that file is created with the appropriate permissions + Args: + path: full path of a file + mode: file permission in octal representation + """ def run_cmd(cmd): if os.geteuid() != 0: cmd.insert(0, 'sudo') @@ -18,5 +26,55 @@ def run_cmd(cmd): run_cmd(['mkdir', '-p', file_path]) if not os.path.isfile(path): run_cmd(['touch', path]) - if (mode is not None): + if (mode is not None): run_cmd(['chmod', mode, path]) + +def cancel_on_sigterm(func): + """ + Wrapper for a function which has to be cancel on SIGTERM + """ + @wraps(func) + def wrapper(*args, **kwargs): + def handler(sig, frame): + if sigterm_handler: + sigterm_handler(sig, frame) + raise Exception("Canceling {}() execution...".format(func.__name__)) + + sigterm_handler = signal.getsignal(signal.SIGTERM) + signal.signal(signal.SIGTERM, handler) + result = None + try: + result = func(*args, **kwargs) + finally: + signal.signal(signal.SIGTERM, sigterm_handler) + return result + return wrapper + +def limit_execution_time(execution_time_secs: int): + """ + Wrapper for a function whose execution time must be limited + Args: + execution_time_secs: maximum execution time in seconds, + after which the function execution will be stopped + """ + def wrapper(func): + @wraps(func) + def execution_func(*args, **kwargs): + def handler(sig, frame): + if sigalrm_handler: + sigalrm_handler(sig, frame) + raise Exception("Canceling {}() execution...".format(func.__name__)) + + sigalrm_handler = signal.getsignal(signal.SIGALRM) + signal.signal(signal.SIGALRM, handler) + signal.alarm(execution_time_secs) + result = None + try: + result = func(*args, **kwargs) + finally: + signal.alarm(0) + + return result + return execution_func + return wrapper + diff --git a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/psu.py b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/psu.py index fb9bce50e071..fbd83d6496ae 100644 --- a/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/psu.py +++ b/platform/barefoot/sonic-platform-modules-bfn-montara/sonic_platform/psu.py @@ -4,18 +4,26 @@ import os import sys import time + import signal + import syslog sys.path.append(os.path.dirname(__file__)) from .platform_thrift_client import thrift_try from sonic_platform_base.psu_base import PsuBase + from platform_utils import cancel_on_sigterm + except ImportError as e: raise ImportError (str(e) + "- required module not found") class Psu(PsuBase): """Platform-specific PSU class""" + sigterm = False + sigterm_default_handler = None + cls_inited = False + def __init__(self, index): PsuBase.__init__(self) self.__index = index @@ -24,6 +32,21 @@ def __init__(self, index): # STUB IMPLEMENTATION self.color = "" + syslog.syslog(syslog.LOG_INFO, "Created PSU #{} instance".format(self.__index)) + if not Psu.cls_inited: + Psu.sigterm_default_handler = signal.getsignal(signal.SIGTERM) + signal.signal(signal.SIGTERM, Psu.signal_handler) + if Psu.sigterm_default_handler: + syslog.syslog(syslog.LOG_INFO, "Default SIGTERM handler overridden!!") + Psu.cls_inited = True + + @classmethod + def signal_handler(cls, sig, frame): + if cls.sigterm_default_handler: + cls.sigterm_default_handler(sig, frame) + syslog.syslog(syslog.LOG_INFO, "Canceling PSU platform API calls...") + cls.sigterm = True + ''' Units of returned info object values: vin - V @@ -33,20 +56,23 @@ def __init__(self, index): fspeed - RPM ''' def __info_get(self): + @cancel_on_sigterm def psu_info_get(client): return client.pltfm_mgr.pltfm_mgr_pwr_supply_info_get(self.__index) # Update cache once per 2 seconds - if self.__ts + 2 < time.time(): + if self.__ts + 2 < time.time() and not Psu.sigterm: self.__info = None try: self.__info = thrift_try(psu_info_get, attempts=1) + except Exception as e: + if "Canceling" in str(e): + syslog.syslog(syslog.LOG_INFO, "{}".format(e)) finally: self.__ts = time.time() return self.__info return self.__info - @staticmethod def get_num_psus(): """ From 179882398c0c035f3bf2a522c07ad5c958453d80 Mon Sep 17 00:00:00 2001 From: Prince George <45705344+prgeor@users.noreply.github.com> Date: Thu, 29 Sep 2022 17:12:20 -0700 Subject: [PATCH 40/51] Revert "Support for serdes platform library debian installation for Innovium SONiC image (#11920)" (#12227) This reverts commit 8c7e0f8e02233dc4236f8f7f048b4281ab9d31b8. --- platform/innovium/invm-sai.mk | 4 +--- platform/innovium/one-image.mk | 2 +- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/platform/innovium/invm-sai.mk b/platform/innovium/invm-sai.mk index 734184e35b0c..25e490e44f83 100755 --- a/platform/innovium/invm-sai.mk +++ b/platform/innovium/invm-sai.mk @@ -5,13 +5,11 @@ INVM_SAI_ONLINE = https://github.com/Innovium/SONiC/raw/master/debian/master INVM_LIBSAI = isai.deb INVM_HSAI = saihdr.deb INVM_DRV = ipd.deb -INVM_SERDES_PLATFORM_LIBRARY = ivm_serdes_pltfm.deb $(INVM_LIBSAI)_URL = $(INVM_SAI_ONLINE)/$(INVM_LIBSAI) $(INVM_HSAI)_URL = $(INVM_SAI_ONLINE)/$(INVM_HSAI) $(INVM_DRV)_URL = $(INVM_SAI_ONLINE)/$(INVM_DRV) -$(INVM_SERDES_PLATFORM_LIBRARY)_URL = $(INVM_SAI_ONLINE)/$(INVM_SERDES_PLATFORM_LIBRARY) $(eval $(call add_conflict_package,$(INVM_HSAI),$(LIBSAIVS_DEV))) -SONIC_ONLINE_DEBS += $(INVM_LIBSAI) $(INVM_HSAI) $(INVM_DRV) $(INVM_SERDES_PLATFORM_LIBRARY) +SONIC_ONLINE_DEBS += $(INVM_LIBSAI) $(INVM_HSAI) $(INVM_DRV) diff --git a/platform/innovium/one-image.mk b/platform/innovium/one-image.mk index 39e2e9fc8246..2cae779d71ac 100755 --- a/platform/innovium/one-image.mk +++ b/platform/innovium/one-image.mk @@ -8,6 +8,6 @@ $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(CEL_MIDSTONE_200I_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(DELTA_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(NETBERG_AURORA_715_PLATFORM_MODULE) $(SONIC_ONE_IMAGE)_LAZY_INSTALLS += $(WISTRON_PLATFORM_MODULE) -$(SONIC_ONE_IMAGE)_INSTALLS += $(INVM_DRV) $(INVM_SERDES_PLATFORM_LIBRARY) +$(SONIC_ONE_IMAGE)_INSTALLS += $(INVM_DRV) $(SONIC_ONE_IMAGE)_DOCKERS += $(SONIC_INSTALL_DOCKER_IMAGES) SONIC_INSTALLERS += $(SONIC_ONE_IMAGE) From 5510d9c03b6f2b1b07198f3c421ab4ce7bf5a0ef Mon Sep 17 00:00:00 2001 From: Ye Jianquan Date: Fri, 30 Sep 2022 08:17:01 +0800 Subject: [PATCH 41/51] Make t0 part1 and part2 be able to be rerun if failed (#12221) Why I did it With continueOnError: true, a failed job returns the result: partiallySuccess, which cause it can't be rerun, since AZP consider it as passed. Then we can't only rerun t0 jobs when it fails. How I did it Mark t0 part1 and part2 as continueOnError: false. How to verify it The pipeline will verify it. --- azure-pipelines.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 7856915f9b60..26a86dffa01e 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -137,7 +137,7 @@ stages: pool: sonictest displayName: "kvmtest-t0-part1" timeoutInMinutes: 360 - continueOnError: true + continueOnError: false steps: - template: .azure-pipelines/run-test-template.yml parameters: @@ -151,7 +151,7 @@ stages: pool: sonictest displayName: "kvmtest-t0-part2" timeoutInMinutes: 360 - continueOnError: true + continueOnError: false steps: - template: .azure-pipelines/run-test-template.yml parameters: From 1d69f0916eccc3961cdaa2c680141f57207396a1 Mon Sep 17 00:00:00 2001 From: Junchao-Mellanox <57339448+Junchao-Mellanox@users.noreply.github.com> Date: Fri, 30 Sep 2022 14:38:05 +0800 Subject: [PATCH 42/51] [Mellanox] Provide dummy implementation for get_rx_los and get_tx_fault (#12231) - Why I did it get_rx_los and get_tx_fault is not supported via the exisitng interface used, need provide dummy implementation for them. NOTE: in later releases we will get them back via different interface. - How I did it Return False * lane_num for get_rx_los and get_tx_fault - How to verify it Added unit test --- .../mlnx-platform-api/sonic_platform/sfp.py | 32 +++++++++++++++++++ .../mlnx-platform-api/tests/test_sfp.py | 14 ++++++++ 2 files changed, 46 insertions(+) diff --git a/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py b/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py index 617b4f33d636..d35b869e9a29 100644 --- a/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py +++ b/platform/mellanox/mlnx-platform-api/sonic_platform/sfp.py @@ -755,6 +755,38 @@ def get_error_description(self): error_description = "Unknow SFP module status ({})".format(oper_status) return error_description + def get_rx_los(self): + """Accessing rx los is not supproted, return all False + + Returns: + list: [False] * channels + """ + api = self.get_xcvr_api() + return [False] * api.NUM_CHANNELS if api else None + + def get_tx_fault(self): + """Accessing tx fault is not supproted, return all False + + Returns: + list: [False] * channels + """ + api = self.get_xcvr_api() + return [False] * api.NUM_CHANNELS if api else None + + def get_xcvr_api(self): + """ + Retrieves the XcvrApi associated with this SFP + + Returns: + An object derived from XcvrApi that corresponds to the SFP + """ + if self._xcvr_api is None: + self.refresh_xcvr_api() + if self._xcvr_api is not None: + self._xcvr_api.get_rx_los = self.get_rx_los + self._xcvr_api.get_tx_fault = self.get_tx_fault + return self._xcvr_api + class RJ45Port(NvidiaSFPCommon): """class derived from SFP, representing RJ45 ports""" diff --git a/platform/mellanox/mlnx-platform-api/tests/test_sfp.py b/platform/mellanox/mlnx-platform-api/tests/test_sfp.py index f599e0241d25..b72a5f3ed4aa 100644 --- a/platform/mellanox/mlnx-platform-api/tests/test_sfp.py +++ b/platform/mellanox/mlnx-platform-api/tests/test_sfp.py @@ -119,3 +119,17 @@ def test_is_port_admin_status_up(self, mock_port_status): mock_port_status.return_value = (0, False) assert not SFP.is_port_admin_status_up(None, None) + + @mock.patch('sonic_platform.sfp.SFP.get_xcvr_api') + def test_dummy_apis(self, mock_get_xcvr_api): + mock_api = mock.MagicMock() + mock_api.NUM_CHANNELS = 4 + mock_get_xcvr_api.return_value = mock_api + + sfp = SFP(0) + assert sfp.get_rx_los() == [False] * 4 + assert sfp.get_tx_fault() == [False] * 4 + + mock_get_xcvr_api.return_value = None + assert sfp.get_rx_los() is None + assert sfp.get_tx_fault() is None From 92bd6dae281977c93bbf39e4456a62857a485dfb Mon Sep 17 00:00:00 2001 From: Volodymyr Samotiy Date: Fri, 30 Sep 2022 09:40:12 +0300 Subject: [PATCH 43/51] [Mellanox] Update SAI to v2205.22.1.19 and SDK/FW to v4.5.3168/v2010.3170 (#12205) - Why I did it To include latest fixes and new functionality SAI fixes and new features fix #3205239, incorrect object type returned for SG child list Fix VRF-VNI map entries remove issue ECC health event and logging [Port Buffers] restore default queue and pg configuration when all user pools are deleted Fix EVPN type3 error on removal of uc/bc flood group Fix EVPN type2 MAC move from local to remote results in SAI failure Fix Disable learning on VXLAN tunnel Fix error on VXLAN v6 tunnel removal Fix port cannot apply schedule group when it is a lag member Fix BFD add more detailed message on BFD packet not related to any existing session gcc10 compilation fixes Disable learning on VXLAN tunnel Support BFD remote-disc exchange in negotiation stage Tunnel Loopback packet action attribute implementation (for Dual TOR) Add KVD resources MIN/MAX functionality (pending CRM issue with MIN only) Support for CRC2 hash algorithm Bulk counter support for PGs, queues Support mirror sample rate attribute (SPC2+) [Functional] [QoS] | Unable to remove SCHEDULE profile table even if there is no object referencing it Next hop group optimized bulk API Reduce verbosity of shared database already exists print Span mirror policer (SPC2+), optimize pipeline for acl mirror action with policer on SPC2+ use same size descriptor pool for rx/tx fix bfd - notify Sonic for admin-down event 2201 - empty list for supported fec for RJ45 ports Fix don't disable used tunnel underlay interfaces SDK fixes 100GbE FCI DAC (10137628-4050LF/HPE PN: 845408-B21) was recognized by mistake as supporting "cable burning' which caused the switch firmware to read page 0x9f (which unsupported in the cable) and to report this cable as having "bad eeprom". Added remote peer UDP port information in BFD packet event. After editing an ECMP, the resilient ECMP next-hop counter may not count correctly. Fixed potential memory leaks in some APIs related to LPM If TTL_CMD_COPY is used in Encap direction for a packet with no TTL, then the value passed in the ttl data structure will be used if non-zero (default 255 if zero). In SN2201: When configuring Force mode, user should configure Speed and FEC on both sides In Flex Tunnel encapsulation flow, if the encapsulation is with an IPv6 header, the flow label field may not be updated as expected. In some cases, when changing speed to 400GbE over 8 lanes, the first few packets would be dropped. In some traffic patterns involving small packets, the PortRcvErrors counter may mistakenly count events of local physical errors due to an internal flow in the hardware that involves link packets. On Spectrum systems, sometimes during link failure, not all previous firmware indications cleared properly, potentially affecting the next link up attempt. On the NVIDIA Spectrum-2 switch, when receiving a packet with Symbol Errors on ports that are configured to cut-thought mode, a pipeline might get stuck. PCI calibration changes from a static to a dynamic mechanism. SDK debug dump shows "Unknown" Counter in RFC3635 Counter Group. SDK debug dump shows "Unknown" Counter in the PPCNT Traffic Class Counter Group. SDK Dump missing column headers in some GC tables may result in difficulty understanding the dump. SLL configuration is missing in SDK dump. Spectrum-2 systems, do no support 1GbE on supported 40GbE modules. When binding a UDP port which is already in use for BFD TX session, the error message appears incorrectly. When Flex Tunnel was used, Flex Modifier sometimes experienced a brief mis-configuration during ISSU. When many ports are active (e.g. 70 ports up), and the configuration of shared buffer is applied on the fly, occasionally, the firmware might get stuck. When running 1GbE speeds on SN4600 system, the port remained active while peer side was closed. When toggling many ports of the Spectrum devices while raising 10GbE link up and link maintenance is enabled, the switch may get stuck and may need to be rebooted. When trying to reconfigure the Flex Parser header and Flex transition parameters after ISSU, the switch will returned an error even if the configuration was identical to that done before performing the ISSU. While toggling the cable, and the low power mode is set to ON, an unexpected PMPE event error is received. - How I did it Updated SDK/SAI submodule and relevant makefiles with the required versions. - How to verify it Build an image and run tests from "sonic-mgmt". Signed-off-by: Volodymyr Samotiy --- platform/mellanox/fw.mk | 6 +++--- platform/mellanox/mlnx-sai.mk | 2 +- platform/mellanox/mlnx-sai/SAI-Implementation | 2 +- platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers | 2 +- platform/mellanox/sdk.mk | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/platform/mellanox/fw.mk b/platform/mellanox/fw.mk index 962ea0ae31e1..92aed0663311 100644 --- a/platform/mellanox/fw.mk +++ b/platform/mellanox/fw.mk @@ -27,17 +27,17 @@ else FW_FROM_URL = n endif -MLNX_SPC_FW_VERSION = 13.2010.2320 +MLNX_SPC_FW_VERSION = 13.2010.3170 MLNX_SPC_FW_FILE = fw-SPC-rel-$(subst .,_,$(MLNX_SPC_FW_VERSION))-EVB.mfa $(MLNX_SPC_FW_FILE)_PATH = $(MLNX_FW_BASE_PATH) $(MLNX_SPC_FW_FILE)_URL = $(MLNX_FW_BASE_URL)/$(MLNX_SPC_FW_FILE) -MLNX_SPC2_FW_VERSION = 29.2010.2320 +MLNX_SPC2_FW_VERSION = 29.2010.3170 MLNX_SPC2_FW_FILE = fw-SPC2-rel-$(subst .,_,$(MLNX_SPC2_FW_VERSION))-EVB.mfa $(MLNX_SPC2_FW_FILE)_PATH = $(MLNX_FW_BASE_PATH) $(MLNX_SPC2_FW_FILE)_URL = $(MLNX_FW_BASE_URL)/$(MLNX_SPC2_FW_FILE) -MLNX_SPC3_FW_VERSION = 30.2010.2320 +MLNX_SPC3_FW_VERSION = 30.2010.3170 MLNX_SPC3_FW_FILE = fw-SPC3-rel-$(subst .,_,$(MLNX_SPC3_FW_VERSION))-EVB.mfa $(MLNX_SPC3_FW_FILE)_PATH = $(MLNX_FW_BASE_PATH) $(MLNX_SPC3_FW_FILE)_URL = $(MLNX_FW_BASE_URL)/$(MLNX_SPC3_FW_FILE) diff --git a/platform/mellanox/mlnx-sai.mk b/platform/mellanox/mlnx-sai.mk index 6eaa1dcf80b4..90ca7430b0ad 100644 --- a/platform/mellanox/mlnx-sai.mk +++ b/platform/mellanox/mlnx-sai.mk @@ -1,6 +1,6 @@ # Mellanox SAI -MLNX_SAI_VERSION = SAIRel1.21.2.0 +MLNX_SAI_VERSION = SAIBuild2205.22.1.19 export MLNX_SAI_VERSION diff --git a/platform/mellanox/mlnx-sai/SAI-Implementation b/platform/mellanox/mlnx-sai/SAI-Implementation index f9a21df71363..82274ffaef77 160000 --- a/platform/mellanox/mlnx-sai/SAI-Implementation +++ b/platform/mellanox/mlnx-sai/SAI-Implementation @@ -1 +1 @@ -Subproject commit f9a21df713636fe648b8bb190698e4494a0f5239 +Subproject commit 82274ffaef7748120b7657362f7875fb7d6e6f5f diff --git a/platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers b/platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers index 5650c3519b55..8b1f1c0f1164 160000 --- a/platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers +++ b/platform/mellanox/sdk-src/sx-kernel/Switch-SDK-drivers @@ -1 +1 @@ -Subproject commit 5650c3519b55051124810a4625f8269694b1e592 +Subproject commit 8b1f1c0f11647749f79ebc4e823c157513067412 diff --git a/platform/mellanox/sdk.mk b/platform/mellanox/sdk.mk index b620b07ee2f8..5a6864bc1e4a 100644 --- a/platform/mellanox/sdk.mk +++ b/platform/mellanox/sdk.mk @@ -16,7 +16,7 @@ # MLNX_SDK_BASE_PATH = $(PLATFORM_PATH)/sdk-src/sx-kernel/Switch-SDK-drivers/bin/ MLNX_SDK_PKG_BASE_PATH = $(MLNX_SDK_BASE_PATH)/$(BLDENV)/$(CONFIGURED_ARCH)/ -MLNX_SDK_VERSION = 4.5.2320 +MLNX_SDK_VERSION = 4.5.3168 MLNX_SDK_ISSU_VERSION = 101 MLNX_SDK_DEB_VERSION = $(subst -,.,$(subst _,.,$(MLNX_SDK_VERSION))) From eea8ebd0a9ee977765097666d1d89961a466d891 Mon Sep 17 00:00:00 2001 From: Volodymyr Samotiy Date: Fri, 30 Sep 2022 09:48:40 +0300 Subject: [PATCH 44/51] [Mellanox] Update MFT to v4.21.0-100 (#11758) - Why I did it To update MFT package to the latest version. - How I did it Updated MFT_VERSION & MFT_REVISION in platform/mellanox/mft.mk. - How to verify it Build an image and deploy to the switch Check MFT version by dpkg -l | grep mft Verify that all the SONiC services up and running Run regression testing using tests from sonic-mgmt Signed-off-by: Volodymyr Samotiy --- platform/mellanox/mft.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/platform/mellanox/mft.mk b/platform/mellanox/mft.mk index eb702a85a1d1..047e87b09086 100644 --- a/platform/mellanox/mft.mk +++ b/platform/mellanox/mft.mk @@ -16,8 +16,8 @@ # # Mellanox SAI -MFT_VERSION = 4.20.0 -MFT_REVISION = 34 +MFT_VERSION = 4.21.0 +MFT_REVISION = 100 export MFT_VERSION MFT_REVISION From 004a8b6eae203cb1496beb8b192b087cc2a32a66 Mon Sep 17 00:00:00 2001 From: Hua Liu <58683130+liuh-80@users.noreply.github.com> Date: Fri, 30 Sep 2022 15:56:46 +0800 Subject: [PATCH 45/51] [AzurePipeline] Fix vstest step failed by libyang missing. (#12240) Why I did it Fix PR merge failed because 'vstest' step does not install libyang. How I did it Install libyang in azure pipeline. How to verify it Pass vstest step. --- azure-pipelines.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 26a86dffa01e..a9004af559ef 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -113,6 +113,8 @@ stages: - script: | set -x + sudo apt-get update + sudo apt-get install libyang0.16 -y sudo dpkg -i --force-confask,confnew ../libswsscommon_1.0.0_amd64.deb sudo dpkg -i ../python3-swsscommon_1.0.0_amd64.deb sudo docker load -i ../target/docker-sonic-vs.gz From 18850e4e28bb5d536abc9c455138c57aad69811d Mon Sep 17 00:00:00 2001 From: Samuel Angebault Date: Fri, 30 Sep 2022 10:03:40 +0200 Subject: [PATCH 46/51] [Arista] Update platform submodules (#12225) Implement input power psu API Report DC power output via API Add bootloader Component in API Fix issue where naming was not unique for Component --- platform/barefoot/sonic-platform-modules-arista | 2 +- platform/broadcom/sonic-platform-modules-arista | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/platform/barefoot/sonic-platform-modules-arista b/platform/barefoot/sonic-platform-modules-arista index e12a04b24c5f..11180c37fa17 160000 --- a/platform/barefoot/sonic-platform-modules-arista +++ b/platform/barefoot/sonic-platform-modules-arista @@ -1 +1 @@ -Subproject commit e12a04b24c5f752a9ca789d62bb7b94c563e1c4b +Subproject commit 11180c37fa17421afdeef346b3896552872a2721 diff --git a/platform/broadcom/sonic-platform-modules-arista b/platform/broadcom/sonic-platform-modules-arista index e12a04b24c5f..11180c37fa17 160000 --- a/platform/broadcom/sonic-platform-modules-arista +++ b/platform/broadcom/sonic-platform-modules-arista @@ -1 +1 @@ -Subproject commit e12a04b24c5f752a9ca789d62bb7b94c563e1c4b +Subproject commit 11180c37fa17421afdeef346b3896552872a2721 From 0a2743d5e472d25209a846e49f032998ee093261 Mon Sep 17 00:00:00 2001 From: jingwenxie Date: Sat, 1 Oct 2022 11:36:55 -0700 Subject: [PATCH 47/51] [submodule] update sonic-utilities (#12138) 0a7557bd9 [minigraph] add option to specify golden path in load_minigraph (#2350) 322aefc37 [GCU]Remove GCU unique lane check for duplicate lanes platforms (#2343) 7099fffa7 [fastboot] fastboot enhancement: Use warm-boot infrastructure for fast-boot (#2286) 09026edbb [warm-reboot] fix warm-reboot when /tmp/cache is missing (#2367) a3c404c74 Fix typo in platform_sfputil_helper.is_rj45_port (#2374) 637d834ce Vnet_route_check Vxlan tunnel route update. (#2281) 29a3e5180 Added support for tunnel route status in show vnet routes all. (#2341) 1ac584bb3 Use 'default' VRF when VRF name is not provided (#2368) 4d377a620 [subinterface]Added additional checks in portchannel and subinterface commands (#2345) bbcdf2ed7 disk_check: Publish event for RO state (#2320) 3fd537b0a Support the bandit check by GitHub Action (#2358) 491d3d380 [generate dump]Added error message when saisdkdump fails (#2356) 6830e01ec [counterpoll]Fixing counterpoll show for tunnel and acl stats (#2355) 3be2ad7de [fast-reboot]Avoid stopping masked services during fast-reboot (#2335) 0e1b0cf20 [GCU] Fix missing backend in dry run (#2347) 676c31bd0 Add verification for override (#2305) 48997c266 Add Password Hardening CLI support (#2338) 414e239ea update unit tests for swap allocator a91a4922f consider swap checking memory in installer f0ce58635 [route_check]: Ignore standalone tunnel routes (#2325) --- src/sonic-utilities | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/sonic-utilities b/src/sonic-utilities index 3af8ba4acc2b..0a7557bd9162 160000 --- a/src/sonic-utilities +++ b/src/sonic-utilities @@ -1 +1 @@ -Subproject commit 3af8ba4acc2bbc77d17be0d67943703021c7d1e1 +Subproject commit 0a7557bd9162eae40f5d4c4f6fbab92dbad7204b From 8c10851c2aebb44099611248f91da350b16df1dc Mon Sep 17 00:00:00 2001 From: Muhammad Danish <88161975+mdanish-kh@users.noreply.github.com> Date: Sun, 2 Oct 2022 11:02:10 +0500 Subject: [PATCH 48/51] Update azure.github.io links to sonic-net.github.io (#12209) Why I did it azure.github.io/SONiC/ no longer works and returns 404 Not Found. Updated it to the correct sonic-net.github.io/SONiC/ --- files/image_config/environment/motd | 2 +- platform/vs/sonic-gns3a.sh | 4 ++-- src/sonic-device-data/README.md | 2 +- src/sonic-host-services-data/README.md | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/files/image_config/environment/motd b/files/image_config/environment/motd index 8562e330fe2c..0d857e5c5f94 100644 --- a/files/image_config/environment/motd +++ b/files/image_config/environment/motd @@ -10,5 +10,5 @@ You are on Unauthorized access and/or use are prohibited. All access and/or use are subject to monitoring. -Help: http://azure.github.io/SONiC/ +Help: https://sonic-net.github.io/SONiC/ diff --git a/platform/vs/sonic-gns3a.sh b/platform/vs/sonic-gns3a.sh index 41e39cd8686a..2a772ce5a332 100644 --- a/platform/vs/sonic-gns3a.sh +++ b/platform/vs/sonic-gns3a.sh @@ -41,9 +41,9 @@ echo " \"category\": \"router\", \"description\": \"SONiC Virtual Switch/Router\", \"vendor_name\": \"SONiC\", - \"vendor_url\": \"https://azure.github.io/SONiC/\", + \"vendor_url\": \"https://sonic-net.github.io/SONiC/\", \"product_name\": \"SONiC\", - \"product_url\": \"https://azure.github.io/SONiC/\", + \"product_url\": \"https://sonic-net.github.io/SONiC/\", \"registry_version\": 3, \"status\": \"experimental\", \"maintainer\": \"SONiC\", diff --git a/src/sonic-device-data/README.md b/src/sonic-device-data/README.md index e8ccad58b819..d9d403758a86 100644 --- a/src/sonic-device-data/README.md +++ b/src/sonic-device-data/README.md @@ -1,4 +1,4 @@ # sonic-device-data Device-specific data for the SONiC project -See the [SONiC Website](http://azure.github.io/SONiC/) for more information about the SONiC project. +See the [SONiC Website](https://sonic-net.github.io/SONiC/) for more information about the SONiC project. diff --git a/src/sonic-host-services-data/README.md b/src/sonic-host-services-data/README.md index 93af66a83d6b..0b9e714932d2 100644 --- a/src/sonic-host-services-data/README.md +++ b/src/sonic-host-services-data/README.md @@ -16,4 +16,4 @@ dpkg-buildpackage -rfakeroot -Tclean --- -See the [SONiC Website](http://azure.github.io/SONiC/) for more information about the SONiC project. +See the [SONiC Website](https://sonic-net.github.io/SONiC/) for more information about the SONiC project. From 44356fa8d758fffd6e023ebfb7e1c8646a81ede1 Mon Sep 17 00:00:00 2001 From: Dror Prital <76714716+dprital@users.noreply.github.com> Date: Sun, 2 Oct 2022 11:34:24 +0300 Subject: [PATCH 49/51] [Mellanox] Add NVIDIA copyright header for NVIDIA added files (#12130) - Why I did it Add NVIDIA Copyright header for new "NVIDIA" files - How I did it Add the copyright header as remark at the head of the file --- .../ACS-MSN2700/buffers_defaults_objects.j2 | 17 +++++++++++++++++ .../buffers_defaults_objects.j2 | 17 +++++++++++++++++ .../Mellanox-SN4700-C128/port_config.ini | 17 +++++++++++++++++ .../Mellanox-SN4700-C128/sai_4700_128x100g.xml | 17 +++++++++++++++++ .../tests/input_platform/__init__.py | 16 ++++++++++++++++ .../tests/input_platform/output_sfp.py | 17 +++++++++++++++++ platform/mellanox/zero_profiles.j2 | 17 +++++++++++++++++ 7 files changed, 118 insertions(+) diff --git a/device/mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/buffers_defaults_objects.j2 b/device/mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/buffers_defaults_objects.j2 index 29a3c74e5233..e8edeca556d1 100644 --- a/device/mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/buffers_defaults_objects.j2 +++ b/device/mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/buffers_defaults_objects.j2 @@ -1,3 +1,20 @@ +{# + Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. + Apache-2.0 + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +#} + {%- macro generate_buffer_pool_and_profiles_with_inactive_ports(port_names_inactive) %} "BUFFER_POOL": { {% if dynamic_mode is not defined and port_names_inactive|length > 0 -%} diff --git a/device/mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/buffers_defaults_objects.j2 b/device/mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/buffers_defaults_objects.j2 index f0b0e3993bd4..6bf657d1fb7b 100644 --- a/device/mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/buffers_defaults_objects.j2 +++ b/device/mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/buffers_defaults_objects.j2 @@ -1,3 +1,20 @@ +{# + Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. + Apache-2.0 + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +#} + {%- macro generate_buffer_pool_and_profiles_with_inactive_ports(port_names_inactive) %} "BUFFER_POOL": { {% if dynamic_mode is not defined and port_names_inactive|length > 0 -%} diff --git a/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/port_config.ini b/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/port_config.ini index d64b66b0b691..0d67f9b366fc 100644 --- a/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/port_config.ini +++ b/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/port_config.ini @@ -1,3 +1,20 @@ +## +## Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. +## Apache-2.0 +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + # name lanes alias index speed Ethernet0 0,1 etp1a 1 100000 Ethernet2 2,3 etp1b 1 100000 diff --git a/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/sai_4700_128x100g.xml b/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/sai_4700_128x100g.xml index f5d49f8b86ab..2575b49f3fa0 100644 --- a/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/sai_4700_128x100g.xml +++ b/device/mellanox/x86_64-mlnx_msn4700-r0/Mellanox-SN4700-C128/sai_4700_128x100g.xml @@ -1,3 +1,20 @@ + + diff --git a/platform/mellanox/mlnx-platform-api/tests/input_platform/__init__.py b/platform/mellanox/mlnx-platform-api/tests/input_platform/__init__.py index e69de29bb2d1..07ebf17a113e 100644 --- a/platform/mellanox/mlnx-platform-api/tests/input_platform/__init__.py +++ b/platform/mellanox/mlnx-platform-api/tests/input_platform/__init__.py @@ -0,0 +1,16 @@ +# +# Copyright (c) 2017-2022 NVIDIA CORPORATION & AFFILIATES. +# Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# diff --git a/platform/mellanox/mlnx-platform-api/tests/input_platform/output_sfp.py b/platform/mellanox/mlnx-platform-api/tests/input_platform/output_sfp.py index 20a09d1b54f6..170b0246430f 100644 --- a/platform/mellanox/mlnx-platform-api/tests/input_platform/output_sfp.py +++ b/platform/mellanox/mlnx-platform-api/tests/input_platform/output_sfp.py @@ -1,3 +1,20 @@ +# +# Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. +# Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + """ module holding the correct values for the sfp_test.py """ diff --git a/platform/mellanox/zero_profiles.j2 b/platform/mellanox/zero_profiles.j2 index a953c18409b2..007f19c83a0a 100644 --- a/platform/mellanox/zero_profiles.j2 +++ b/platform/mellanox/zero_profiles.j2 @@ -1,3 +1,20 @@ +{# + Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. + Apache-2.0 + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +#} + [ { "BUFFER_POOL_TABLE:ingress_zero_pool": { From 2f46689a059d3d7146b8ab27a0293e4b42383ec7 Mon Sep 17 00:00:00 2001 From: andywongarista <78833093+andywongarista@users.noreply.github.com> Date: Sun, 2 Oct 2022 22:53:34 -0700 Subject: [PATCH 50/51] [Arista] Add components for 720DT-48S (#12217) Why I did it Add components data for sonic-mgmt testing How I did it Update platform.json and add platform_components.json How to verify it Ran sonic-mgmt tests (test_chassis and test_component) --- device/arista/x86_64-arista_720dt_48s/platform.json | 9 ++++++++- .../x86_64-arista_720dt_48s/platform_components.json | 10 ++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) create mode 100644 device/arista/x86_64-arista_720dt_48s/platform_components.json diff --git a/device/arista/x86_64-arista_720dt_48s/platform.json b/device/arista/x86_64-arista_720dt_48s/platform.json index ce405b82f955..c560f8d739f1 100644 --- a/device/arista/x86_64-arista_720dt_48s/platform.json +++ b/device/arista/x86_64-arista_720dt_48s/platform.json @@ -1,7 +1,14 @@ { "chassis": { "name": "CCS-720DT-48S", - "components": [], + "components": [ + { + "name": "Aboot()" + }, + { + "name": "Scd(addr=0000:00:18.7)" + } + ], "fan_drawers": [ { "name": "fixed1", diff --git a/device/arista/x86_64-arista_720dt_48s/platform_components.json b/device/arista/x86_64-arista_720dt_48s/platform_components.json new file mode 100644 index 000000000000..ea8bbb5e3346 --- /dev/null +++ b/device/arista/x86_64-arista_720dt_48s/platform_components.json @@ -0,0 +1,10 @@ +{ + "chassis": { + "CCS-720DT-48S": { + "component": { + "Aboot()": {}, + "Scd(addr=0000:00:18.7)": {} + } + } + } +} From 95f4af3407d32cdec98a3d0d487ab0997ef5ff3c Mon Sep 17 00:00:00 2001 From: Mai Bui Date: Mon, 3 Oct 2022 11:38:55 -0700 Subject: [PATCH 51/51] [actions] Support Semgrep by Github Actions (#12249) Signed-off-by: maipbui #### Why I did it [Semgrep](https://github.com/returntocorp/semgrep) is a static analysis tool to find security vulnerabilities. When opening a PR or commtting to PR, Semgrep performs a diff-aware scanning, which scans changed files in PRs. When merging PR, Semgrep performs a full scan on master branch and report all findings. Ref: - [Supported Language](https://semgrep.dev/docs/supported-languages/#language-maturity) - [Semgrep Rules](https://registry.semgrep.dev/rule) #### How I did it Integrate Semgrep into this repository by committing a job configuration file #### How to verify it PR: https://github.com/maipbui/sonic-buildimage/pull/2 Master branch full scan findings: [Master branch findings results](https://github.com/maipbui/sonic-buildimage/actions/runs/3160181876/jobs/5144332404) PR https://github.com/maipbui/sonic-buildimage/pull/2 scan findings: [Pull request findings results](https://github.com/maipbui/sonic-buildimage/actions/runs/3160193505/jobs/5144357859) --- .github/workflows/semgrep.yml | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 .github/workflows/semgrep.yml diff --git a/.github/workflows/semgrep.yml b/.github/workflows/semgrep.yml new file mode 100644 index 000000000000..8ebe082f50a4 --- /dev/null +++ b/.github/workflows/semgrep.yml @@ -0,0 +1,21 @@ +name: Semgrep + +on: + pull_request: {} + push: + branches: + - master + - '201[7-9][0-1][0-9]' + - '202[0-9][0-1][0-9]' + +jobs: + semgrep: + name: Semgrep + runs-on: ubuntu-latest + container: + image: returntocorp/semgrep + steps: + - uses: actions/checkout@v3 + - run: semgrep ci + env: + SEMGREP_RULES: p/default