Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7260cx3 DualToR config.bcm support based on DualToR setting in device metadata at boot time #7168

Merged
merged 4 commits into from
Mar 31, 2021

Conversation

gechiang
Copy link
Collaborator

Why I did it

In order to support DualToR IPinIP functionality for 7260CX3 SKUs there are new BRCM SOC properties required. We do not want these new SOC properties to be applied for all 7260CX3 SKUs but to base on the DUT role to decide which DUT should include these new SOC properties at boot time.

How I did it

Added code to check the device metadata to inspect if the DUT is configured for DualToR functionality. Renamed/Updated the current config.bcm file of the supported 7260cx3 SKUs to optionally include the IPinIP required SOC properties based on the DUT DualToR role. This file is used for "sonic-cfggen" at boot time to dynamically generate the required config.bcm file to be used by BRCM SDK. This new method eliminated the need to have multiple versions of config.bcm files where the delta is small.

NOTE:

This PR has pulled in the changes made by (#7027)
and converted the config.bcm selection to use this new method since the MMU selection SOC property is the only line of difference between the T0 and T1 config.bcm files based on the switch role being T0 or T1.

How to verify it

Manually changed the device metadata to either include or don't include the DualToR role and verify that the correct bcm.config is installed in the ASIC.
Here is the verification output for the case where the 7260cx3 is not configured for DualToR purpose:

admin@str2-7260cx3-acs-9:~$ docker exec -it syncd bash
root@str2-7260cx3-acs-9:/# cd /etc/sai.d/
root@str2-7260cx3-acs-9:/etc/sai.d# ls -l
total 40
-rw-r--r-- 1 root root 36437 Mar 26 05:20 sai.config
-rw-r--r-- 1 root root    67 Mar 26 05:20 sai.profile
root@str2-7260cx3-acs-9:/etc/sai.d# cat sai.profile
SAI_INIT_CONFIG_FILE=/etc/sai.d/sai.config
SAI_NUM_ECMP_MEMBERS=64
root@str2-7260cx3-acs-9:/etc/sai.d# cat sai.config

PHY_AN_ALLOW_PLL_CHANGE=1
arl_clean_timeout_usec=15000000
asf_mem_profile=2
bcm_num_cos=10
bcm_stat_flags=1
bcm_stat_jumbo=9236
cdma_timeout_usec=15000000
core_clock_frequency=1700
dma_desc_timeout_usec=15000000
dpp_clock_ratio=2:3
higig2_hdr_mode=1
ipv6_lpm_128b_enable=1
l3_alpm_enable=2
lpm_scaling_enable=0
l2xmsg_mode=1
max_vp_lags=0
mem_scan_enable=1
miim_intr_enable=0
module_64ports=1
os=unix
oversubscribe_mode=1
pbmp_xport_xe=0xfffffffd3fffffff4fffffffc7ffffffe
#pbmp_xport_xe=0x47fffffff1fffffffcfffffffe7ffffffe
phy_an_allow_pll_change_hg=0
phy_an_c73=1
phy_chain_rx_lane_map_physical{1.0}=0x3210
phy_chain_rx_lane_map_physical{101.0}=0x0123
...

Here is the validation for the case where a 7260CX3 do have DualToR configure:

admin@str2-7260cx3-acs-9:~$ bcmcmd "config show" | grep sai_
    sai_tunnel_support=1
admin@str2-7260cx3-acs-9:~$ bcmcmd "config show" | grep ecmp
    l3_ecmp_levels=2
admin@str2-7260cx3-acs-9:~$ bcmcmd "config show" | grep host_as_r
    host_as_route_disable=1

admin@str2-7260cx3-acs-9:~$ docker exec -it syncd bash
root@str2-7260cx3-acs-9:/# cd /etc/sai.d/
root@str2-7260cx3-acs-9:/etc/sai.d# ls -l
total 40
-rw-r--r-- 1 root root 36534 Mar 26 05:32 sai.config
-rw-r--r-- 1 root root    67 Mar 26 05:32 sai.profile
root@str2-7260cx3-acs-9:/etc/sai.d# cat sai.profile
SAI_INIT_CONFIG_FILE=/etc/sai.d/sai.config
SAI_NUM_ECMP_MEMBERS=64
root@str2-7260cx3-acs-9:/etc/sai.d# cat sai.config
sai_tunnel_support=1
                               host_as_route_disable=1
                               l3_ecmp_levels=2
mmu_init_config="MSFT-TH2-Tierr0"
PHY_AN_ALLOW_PLL_CHANGE=1
arl_clean_timeout_usec=15000000
asf_mem_profile=2
bcm_num_cos=10
bcm_stat_flags=1
bcm_stat_jumbo=9236
cdma_timeout_usec=15000000
core_clock_frequency=1700
dma_desc_timeout_usec=15000000
dpp_clock_ratio=2:3
higig2_hdr_mode=1
ipv6_lpm_128b_enable=1
l3_alpm_enable=2
lpm_scaling_enable=0
l2xmsg_mode=1
max_vp_lags=0
mem_scan_enable=1
miim_intr_enable=0
module_64ports=1
os=unix
oversubscribe_mode=1
pbmp_xport_xe=0xfffffffd3fffffff4fffffffc7ffffffe
#pbmp_xport_xe=0x47fffffff1fffffffcfffffffe7ffffffe
phy_an_allow_pll_change_hg=0
phy_an_c73=1
phy_chain_rx_lane_map_physical{1.0}=0x3210
phy_chain_rx_lane_map_physical{101.0}=0x0123
phy_chain_rx_lane_map_physical{105.0}=0x0123
phy_chain_rx_lane_map_physical{109.0}=0x0123
...

Here is the result when the same code is running on a non 7260cx3 SKU where the config .bcm setup remains unchanged and continue to load from the original config.bcm file without going through "sonic-cfggen":

admin@str-s6000-acs-8:~$ docker exec -it syncd bash
root@str-s6000-acs-8:/# cd /etc/sai.d/
root@str-s6000-acs-8:/etc/sai.d# ls -l
total 4
-rw-r--r-- 1 root root 96 Mar 26 05:22 sai.profile
root@str-s6000-acs-8:/etc/sai.d# cat sai.profile
SAI_INIT_CONFIG_FILE=/usr/share/sonic/hwsku/td2-s6000-32x40G.config.bcm
SAI_NUM_ECMP_MEMBERS=32
root@str-s6000-acs-8:/etc/sai.d#

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012

Description for the changelog

7260cx3 dynamic generate config.bcm based on DualToR role at boot time to add IPinIP support.

Copy link
Collaborator

@lguohan lguohan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕐

@gechiang gechiang requested a review from lguohan March 28, 2021 06:39
Copy link
Collaborator

@lguohan lguohan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please wait for ying approval as well.

lguohan
lguohan previously approved these changes Mar 29, 2021
@gechiang
Copy link
Collaborator Author

@yxieca Appreciate if you can take a look at my changes to see if any changes required.
Thanks!

@gechiang gechiang merged commit 6f65b42 into sonic-net:master Mar 31, 2021
daall pushed a commit that referenced this pull request Apr 15, 2021
… metadata at boot time (#7168)

* 7260cx3 DualToR config.bcm support based on DualToR setting in device metadata at boot time. 
For HWSKU Arista-7260CX3-C64 the MMU setting SOC for T0/T1 is also combined into the config.bcm.j2 logic so use just one config file and adding delta based on Switch Roles.
raphaelt-nvidia pushed a commit to raphaelt-nvidia/sonic-buildimage that referenced this pull request May 23, 2021
… metadata at boot time (sonic-net#7168)

* 7260cx3 DualToR config.bcm support based on DualToR setting in device metadata at boot time. 
For HWSKU Arista-7260CX3-C64 the MMU setting SOC for T0/T1 is also combined into the config.bcm.j2 logic so use just one config file and adding delta based on Switch Roles.
carl-nokia pushed a commit to carl-nokia/sonic-buildimage that referenced this pull request Aug 7, 2021
… metadata at boot time (sonic-net#7168)

* 7260cx3 DualToR config.bcm support based on DualToR setting in device metadata at boot time. 
For HWSKU Arista-7260CX3-C64 the MMU setting SOC for T0/T1 is also combined into the config.bcm.j2 logic so use just one config file and adding delta based on Switch Roles.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants