[201911]: Adding SKU Mellanox-SN3800-D100C12S2 #15

madhanmellanox · 2021-06-24T03:00:16Z

Why I did it

To create a new SKU Mellanox-SN3800-D100C12S2

How I did it

I arrived at the SKU configuration values based on the following SKU template, Port mapping and number of uplinks and downlinks.

SKU template:
Port configuration
• Breakout mode for each port - Defined in port mapping
• Speed of the port Defined in Port mapping
• Auto-negotiation enable/disable No setting required
• FEC mode No setting required
• Type of transceiver used Not needed
Buffer configuration
• Shared headroom enable
• If shared headroom enabled what is the over-subscription ratio as in SN3800
• Dynamic Buffer disable
• In static buffer scenario how many uplinks and downlinks? as in SN3800
• 2km cable support required? no
Switch configuration
• Warmboot enabled? yes
• Should warmboot be added to SAI profile when enabled? yes
• Is VxLAN source port range set? yes
• Should Vxlan source port range be added to SAI profile when set. as in SN3800
• Is Static Policy Based Hashing enabled? no

Port Mapping
etp1 to etp36 split into 50G
etp37 and etp40 is 10G
etp38 and etp39 splint into 50G
etp41 to etp52 split into 50G
etp53 to etp64 is 100G

Number of Uplinks / Downlinks:
TO topology: 12 100G uplinks and rest all downlinks.
T1 topology: (SKU will not be used in T1 topology), so same 12 100G uplinks and rest all downlinks is used to arrive at buffer config values.

How to verify it

Build the image, install it on the 3800 switch and set the SKU and verify the ports come up with proper speeds.

Which release branch to backport (provide reason below if selected)

201811
201911
202006
202012

Description for the changelog

Changes are in sonic-buildimage/device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/ folder.

A picture of a cute animal (not mandatory but encouraged)

stephenxs · 2021-06-24T08:24:45Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/pg_profile_lookup.ini

@@ -0,0 +1,17 @@
+#  PG lossless profiles.


We don't need to add a new file. We can just point it to the one in Mellanox-SN3800-D112C8

stephenxs · 2021-06-24T08:26:00Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t0.j2

@@ -0,0 +1,100 @@
+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '47194112' %}
+{% set ingress_lossless_pool_xoff =  '47194112' %}


The xoff stands for shared headroom pool size. It should not be equal to pool size.
The buffer pool size is not correct. It's too large for Spectrum 2. You probably used Spectrum 3.
My result is 20664320 xoff 3321856

10G 5m * 2 headroom 44kb

50G 5m * 100 headroom 51kb

100G 40m * 12 headroom 66kb

stephenxs · 2021-06-24T08:26:29Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t1.j2

@@ -0,0 +1,100 @@
+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '46133248' %}
+{% set ingress_lossless_pool_xoff =  '46133248' %}


Thanks Stephen, I will incorporate these changes.

stephenxs · 2021-06-24T13:01:49Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t0.j2

@@ -0,0 +1,100 @@
+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '20664320' %}
+{% set ingress_lossless_pool_xoff =  '20664320' %}


Suggested change

{% set ingress_lossless_pool_xoff = '20664320' %}

{% set ingress_lossless_pool_xoff = '3321856' %}

stephenxs · 2021-06-24T13:02:58Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t0.j2

+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '20664320' %}
+{% set ingress_lossless_pool_xoff =  '20664320' %}
+{% set egress_lossless_pool_size =  '3321856' %}


Suggested change

{% set egress_lossless_pool_size = '3321856' %}

{% set egress_lossless_pool_size = '34287552' %}

stephenxs · 2021-06-24T13:11:43Z

ingress_lossless_pool_xoff represents the shared headroom pool size.
egress_lossless_pool_size should be the MMUSize

stephenxs · 2021-06-24T13:13:08Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t1.j2

@@ -0,0 +1,100 @@
+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '19601408' %}
+{% set ingress_lossless_pool_xoff =  '19601408' %}


Suggested change

{% set ingress_lossless_pool_xoff = '19601408' %}

{% set ingress_lossless_pool_xoff = '4384768' %}

stephenxs · 2021-06-24T13:13:28Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/buffers_defaults_t1.j2

+{% set default_cable = '5m' %}
+{% set ingress_lossless_pool_size =  '19601408' %}
+{% set ingress_lossless_pool_xoff =  '19601408' %}
+{% set egress_lossless_pool_size =  '4384768' %}


Suggested change

{% set egress_lossless_pool_size = '4384768' %}

{% set egress_lossless_pool_size = '34287552' %}

stephenxs

Approved for buffer confurations

dgsudharsan · 2021-06-24T16:28:26Z

device/mellanox/x86_64-mlnx_msn3800-r0/Mellanox-SN3800-D100C12S2/port_config.ini

+Ethernet0    0,1              etp1a          1    50000
+Ethernet2    2,3              etp1b          1    50000
+Ethernet4    4,5              etp2a          2    50000
+Ethernet4    6,7              etp2b          2    50000


Why do we have two Ethernet4 defined here? I believe it should be Ethernet6

got it @dgsudharsan, will fix it.

volodymyrsamotiy · 2021-06-24T16:40:07Z

@madhanmellanox, please fix comment provided by @dgsudharsan.
Everything else looks good to me.

dgsudharsan

Approving it for SONiC port level changes.

…er mode of 10G ports

stephenxs · 2021-06-25T01:19:33Z

Currently, we don't take port admin state into account when calculating buffer pool size.
There is a request about reclaiming the reserved buffer of admin down port from MSFT, which is under internal discussion.
I suggest taking the number of all ports instead of admin up ports into account when calculating the buffer sizes for now, which means we don't need to update the buffer pool size.

dgsudharsan · 2021-11-03T01:17:06Z

@madhanmellanox Can you please close this?

[201911]: Adding SKU Mellanox-SN3800-D100C12S2

6556ff2

stephenxs suggested changes Jun 24, 2021

View reviewed changes

modified buffers defaults t0 and t1 config values

f883fd9

stephenxs suggested changes Jun 24, 2021

View reviewed changes

stephenxs reviewed Jun 24, 2021

View reviewed changes

modified buffer values and made pg_profile_lookup.ini as a symbolic file

79c1da3

stephenxs approved these changes Jun 24, 2021

View reviewed changes

dgsudharsan suggested changes Jun 24, 2021

View reviewed changes

Addressed comments in port_config.ini

26bccc1

volodymyrsamotiy approved these changes Jun 24, 2021

View reviewed changes

modifing SAI XML file value in SAI Profile file

d9e813f

dgsudharsan approved these changes Jun 24, 2021

View reviewed changes

Madhan Babu added 2 commits June 24, 2021 23:39

fixed index value of 39 for etp39a and etp39b ports in port_config.ini

8718c1e

changed buffer config values due to removing 4 50G ports due to retim…

8888395

…er mode of 10G ports

Madhan Babu added 2 commits June 25, 2021 04:53

reverted back buffer values to previous values accounting all the ports

1ea2d7b

fixed the SAI XML file for 50G with split 2 tag

345caf8

stephenxs approved these changes Jun 25, 2021

View reviewed changes

modified 10G ports according to HW restrictions

fd1607c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[201911]: Adding SKU Mellanox-SN3800-D100C12S2 #15

[201911]: Adding SKU Mellanox-SN3800-D100C12S2 #15

madhanmellanox commented Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs Jun 24, 2021

madhanmellanox Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs commented Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs Jun 24, 2021

stephenxs left a comment

dgsudharsan Jun 24, 2021 •

edited

Loading

madhanmellanox Jun 24, 2021

volodymyrsamotiy commented Jun 24, 2021

dgsudharsan left a comment

stephenxs commented Jun 25, 2021

dgsudharsan commented Nov 3, 2021

	{% set ingress_lossless_pool_xoff = '20664320' %}
	{% set ingress_lossless_pool_xoff = '3321856' %}

	{% set egress_lossless_pool_size = '3321856' %}
	{% set egress_lossless_pool_size = '34287552' %}

	{% set ingress_lossless_pool_xoff = '19601408' %}
	{% set ingress_lossless_pool_xoff = '4384768' %}

	{% set egress_lossless_pool_size = '4384768' %}
	{% set egress_lossless_pool_size = '34287552' %}

[201911]: Adding SKU Mellanox-SN3800-D100C12S2 #15

Are you sure you want to change the base?

[201911]: Adding SKU Mellanox-SN3800-D100C12S2 #15

Conversation

madhanmellanox commented Jun 24, 2021

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephenxs commented Jun 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephenxs left a comment

Choose a reason for hiding this comment

dgsudharsan Jun 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

volodymyrsamotiy commented Jun 24, 2021

dgsudharsan left a comment

Choose a reason for hiding this comment

stephenxs commented Jun 25, 2021

dgsudharsan commented Nov 3, 2021

dgsudharsan Jun 24, 2021 •

edited

Loading