[multi-asic]: Add multi-asic virtual switch testbed #2168

SuvarnaMeenakshi · 2020-09-01T20:57:04Z

Description of PR

Summary:
Fixes # (issue)

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

What is the motivation for this PR?

Add new testbed for multi-asic VS platform with HwSKU: msft_multi_asic_vs

How did you do it?

To create a multi-asic VS testbed, the assumption is that the vs image is a multi-asic VS, which means that the asic.conf is updated with the right number of asics. In this PR, multi-asic VS of SKU: msft_multi_asic_vs is used with number of ASICS as 6.The below changes will address the modification:

Add new HwSKU msft_multi_asic_vs with 64 front-panel interfaces.
Add new testbed information.
Modify sonic.xml.j2 to have higher vcpu and vmemory for multi-asic vs.
Add num_asics parameter in veos_vtb to denote number of asics of that device.
Update port_alias to parse port_config files for each asic for multi-asic device and retrieve external interfaces so that VS image can come up with required number of external interfaces.

How did you verify/test it?

Bring up vs testbed using the command: ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-multi-asic-t1-lag password.txt
Ensure that the sonic-vs testbed comes up fine. Sonic vm should be accessible via console access. Configuration changes are not added in this PR.

Ensure that single-asic vs testbed comes up fine using:
./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k ceos add-topo vms-kvm-t0 password.txt
./testbed-cli.sh -t vtestbed.csv -m veos_vtb deploy-mg vms-kvm-t0 lab password.txt

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Add a new parameter in port_alias.py to retrieve only external interfaces. Remove incorrect changes. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

multi-asic vs. - Add num_asic parameter in veos_vtb to denote number of asics of that device. - Update port_alias to get port_config files for each asic for multi-asic device. - Update sonic-kickstart to add number of asics in asic.conf file. - Reboot sonic-vs number of asics is more than 1 so that the vs comes up as multi-asic vs. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

of minigraph. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

ansible/library/port_alias.py

…vs_check Signed-off-by: Ubuntu <sumeenak@sumeenak-ubuntu-vm0.q3q211rtov2etb23rce1vewych.cx.internal.cloudapp.net>

Signed-off-by: Ubuntu <sumeenak@sumeenak-ubuntu-vm0.q3q211rtov2etb23rce1vewych.cx.internal.cloudapp.net>

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

wangxin · 2020-11-05T08:25:44Z

ansible/vtestbed.csv

@@ -4,3 +4,4 @@ vms-kvm-t0-64,vms6-1,t0-64,docker-ptf,ptf-01,10.250.0.102/24,fec0::ffff:afa:2/64
 vms-kvm-t1-lag,vms6-2,t1-lag,docker-ptf,ptf-02,10.250.0.106/24,fec0::ffff:afa:6/64,server_1,VM0104,[vlab-03],Tests virtual switch vm
 vms-kvm-t0-2,vms6-3,t0,docker-ptf,ptf-03,10.250.0.108/24,fec0::ffff:afa:8/64,server_1,VM0104,[vlab-04],Tests virtual switch vm
 vms-kvm-dual-t0,vms6-4,dualtor,docker-ptf,ptf-04,10.250.0.109/24,fec0::ffff:afa:9/64,server_1,VM0108,[vlab-05;vlab-06],Dual-TOR testbed
+vms-kvm-multi-asic-t1-lag,vms6-4,t1-lag,docker-ptf,ptf-unknown,10.250.0.106/24,fec0::ffff:afa:6/64,server_1,VM0104,[vlab-07],Tests multi-asic virtual switch vm


I think a new PTF should be defined for this multi-ASIC setup. For example:

ptf-05

10.250.0.109

fec0::ffff:afa:a/64

The ptf-05 host also should be added to ansible/veos_vtb.

Added new ptf-docker as suggested.

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi · 2020-11-05T18:29:52Z

retest vsimage please

wangxin · 2020-11-06T03:13:39Z

retest vsimage please

SuvarnaMeenakshi · 2020-11-06T17:01:58Z

retest vsimage please

abdosi · 2020-11-07T00:23:01Z

ansible/library/port_alias.py

+        else:
+            num_asic = allmap.get_num_asic()
+        if num_asic == 1:
+            (aliases, portmap, aliasmap, portspeed) = allmap.get_portmap(all_ports)


remove this if else.. for loop should be enough.

abdosi · 2020-11-07T00:32:50Z

ansible/library/port_alias.py

+        if m_args['num_asic'] is not None:
+            num_asic = m_args['num_asic']
+        else:
+            num_asic = allmap.get_num_asic()


use sonic-py-common utility here

Modified to use sonic_py_common utility.
But as port_alias.py script is invoked from sonic_mgmt docker during VS bring up or is run on the device/VS during minigraph generation, the import of sonic_py_common library cannot be done globally as it will not present when run on sonic-mgmt docker.

abdosi · 2020-11-07T00:33:05Z

ansible/library/port_alias.py

        if os.path.exists(portconfig):
            return portconfig
        return None

-    def get_portmap(self):
+    def get_num_asic(self):
+        num_asic = 1


we can remove this.

abdosi · 2020-11-07T00:33:36Z

ansible/roles/vm_set/library/sonic_kickstart.py

@@ -117,6 +117,7 @@ def session(new_params):
        ('ip route add 0.0.0.0/0 via %s table default' % str(new_params['mgmt_gw']), [r'#']),
        ('ip route', [r'#']),
        ('echo %s:%s | chpasswd' % (str(new_params['login']), str(new_params['new_password'])), [r'#']),
+        ('echo NUM_ASIC=%s > /usr/share/sonic/device/x86_64-kvm_x86_64-r0/asic.conf' % (str(new_params['num_asic'])), [r'#']),


will not be needed

abdosi · 2020-11-07T00:33:53Z

ansible/roles/vm_set/tasks/start_sonic_vm.yml

  register: kickstart_output

 - name: Fail if kickstart gives error for {{ dut_name }}
  fail: msg="Start sonic vm weren't succesfull"
  when: kickstart_output.kickstart_code != 0
+
+- name: Reboot sonic-vm to bring up as multi-asic vs {{ num_asic }}
+  command: "virsh reboot {{ dut_name }}"


Removed as we can use a pre-built sonic-vs image where number of asics is updated in asic.conf.

- Use a pre build sonic multiasic VS image with asic.conf updated with number of asics. - Remove updation on asic.conf during multiasic VS testbed image bring up. - Remove function added to retrieve number of asics in port_alias. When executed from DUT, use sonic_py_common function to retrieve number of asics. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

wangxin · 2020-11-23T09:34:15Z

retest vsimage please

exception on import error. Remove all_ports variable added, will always retrieve external ports alone. Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

ansible/library/port_alias.py

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

ansible/library/port_alias.py

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

For multi-asic platform, there will be multiple swss services running. So instead of checking for each swss services, check if all services are up and then proceed with configuring eth0 IP address. Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

judyjoseph · 2021-01-11T18:54:45Z

ansible/roles/vm_set/library/sonic_kickstart.py

@@ -108,7 +108,7 @@ def logout(self):

 def session(new_params):
    seq = [
-        ('while true; do if [ $(systemctl is-active swss) == "active" ]; then break; fi; echo $(systemctl is-active swss); sleep 1; done', [r'#'], 180),
+        ('while true; do if [ $(systemctl is-system-running) == "degraded" ]; then break; fi; echo $(systemctl is-system-running); sleep 1; done', [r'#'], 180),


Can you add the check for the state "is-system-running" == "running" also ? It could go into this state as well.
https://www.freedesktop.org/software/systemd/man/systemctl.html.

before assigning eth0 IP address. As there is no default configuration for multi-asic VS, interfaces-config will nto override the configured eth0 IP address so we do not need to check for interfaces-config service completion for multi-asic VS. No change in behavior for single asic VS. Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi added 2 commits August 31, 2020 12:07

[multi_asic_vs]: Create new testbed for multi_asic vs.

0f294f9

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Minor fix.

d2b2b80

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi requested a review from arlakshm September 1, 2020 20:57

SuvarnaMeenakshi added 7 commits September 4, 2020 09:41

Increase memory and vcpu for multi-asic vs in sonic.xml.

6da37ee

Add a new parameter in port_alias.py to retrieve only external interfaces. Remove incorrect changes. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Remove changes in minigraph_facts.

c5d1c74

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Minor fix.

64eb686

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Minor fix.

146a674

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Fix port_alias to make sure it is able to run during configuration

c674d2f

of minigraph. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Remove test changes.

4ab122f

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi marked this pull request as ready for review September 11, 2020 01:43

SuvarnaMeenakshi requested review from lguohan, judyjoseph and abdosi September 11, 2020 01:43

wangxin reviewed Sep 16, 2020

View reviewed changes

ansible/library/port_alias.py Outdated Show resolved Hide resolved

wangxin approved these changes Sep 17, 2020

View reviewed changes

yxieca force-pushed the master branch from 885053e to 8f9cb56 Compare October 31, 2020 02:57

Ubuntu added 3 commits November 4, 2020 18:15

Merge remote-tracking branch 'remotes/origin/master' into multi_asic_…

58e3814

…vs_check Signed-off-by: Ubuntu <sumeenak@sumeenak-ubuntu-vm0.q3q211rtov2etb23rce1vewych.cx.internal.cloudapp.net>

Add multi-asic related child nodes in inventory files.

3b1e319

Signed-off-by: Ubuntu <sumeenak@sumeenak-ubuntu-vm0.q3q211rtov2etb23rce1vewych.cx.internal.cloudapp.net>

Fix symlink.

3778441

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi requested a review from wangxin November 5, 2020 02:17

wangxin requested changes Nov 5, 2020

View reviewed changes

Add a new ptf-docker for multi-asic VS testbed.

05c3a64

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

wangxin approved these changes Nov 6, 2020

View reviewed changes

abdosi reviewed Nov 7, 2020

View reviewed changes

Ubuntu added 4 commits November 17, 2020 22:57

Merge remote-tracking branch 'origin/master' into multi_asic

20e4446

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Remove empty lines.

b1bf298

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

Fix spacing.

adb81e5

Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

SuvarnaMeenakshi added 2 commits December 14, 2020 21:51

Updated port_alias to import multi_asic library and handle

1bc85e9

exception on import error. Remove all_ports variable added, will always retrieve external ports alone. Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

Minor fix.

e2030b3

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

judyjoseph reviewed Dec 16, 2020

View reviewed changes

ansible/library/port_alias.py Show resolved Hide resolved

Fix topology for multi_asic_vs testbed.

f7e18d8

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

judyjoseph reviewed Dec 16, 2020

View reviewed changes

ansible/library/port_alias.py Show resolved Hide resolved

SuvarnaMeenakshi added 4 commits December 16, 2020 21:02

Minor fix based on review comment.

51465b6

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

Remove previous wrong commit.

5e63595

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>

Merge remote-tracking branch 'origin/master' into multi_asic_vs_initial

798595d

judyjoseph reviewed Jan 11, 2021

View reviewed changes

judyjoseph approved these changes Jan 20, 2021

View reviewed changes

abdosi approved these changes Jan 21, 2021

View reviewed changes

SuvarnaMeenakshi merged commit 4f4c6f1 into sonic-net:master Jan 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[multi-asic]: Add multi-asic virtual switch testbed #2168

[multi-asic]: Add multi-asic virtual switch testbed #2168

SuvarnaMeenakshi commented Sep 1, 2020 •

edited

Loading

wangxin Nov 5, 2020 •

edited

Loading

SuvarnaMeenakshi Nov 5, 2020

SuvarnaMeenakshi commented Nov 5, 2020

wangxin commented Nov 6, 2020

SuvarnaMeenakshi commented Nov 6, 2020

abdosi Nov 7, 2020

abdosi Nov 7, 2020

SuvarnaMeenakshi Nov 17, 2020

abdosi Nov 7, 2020

SuvarnaMeenakshi Nov 17, 2020

abdosi Nov 7, 2020

SuvarnaMeenakshi Nov 17, 2020

abdosi Nov 7, 2020

SuvarnaMeenakshi Nov 17, 2020

wangxin commented Nov 23, 2020

judyjoseph Jan 11, 2021

[multi-asic]: Add multi-asic virtual switch testbed #2168

[multi-asic]: Add multi-asic virtual switch testbed #2168

Conversation

SuvarnaMeenakshi commented Sep 1, 2020 • edited Loading

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

wangxin Nov 5, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SuvarnaMeenakshi commented Nov 5, 2020

wangxin commented Nov 6, 2020

SuvarnaMeenakshi commented Nov 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangxin commented Nov 23, 2020

Choose a reason for hiding this comment

SuvarnaMeenakshi commented Sep 1, 2020 •

edited

Loading

wangxin Nov 5, 2020 •

edited

Loading