[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. #3110

bingwang-ms · 2021-03-09T07:50:24Z

Signed-off-by: bingwang [email protected]

Description of PR

Summary:
Fixes #3256
This PR implements dualtor t1->standby_tor orchagent test cases.
Test plan https://github.com/Azure/sonic-mgmt/blob/master/docs/testplan/dual_tor/dual_tor_orch_test_plan.md#test-cases
T1 -> Standby ToR

Send traffic of varying tuple destined to server under standby mux. The following are the various steps and those which have to be executed in a sequence is grouped together.

Step	Goal	Expected results
All ports to T1s are up; Loopback route configured	ECMP hashing	Verify tunnel traffic to Active ToR is distributed equally across nexthops; Verify no traffic is forwarded to downlink in case of standby mux
Shutdown one uplink to T1	ECMP hashing/CRM	Verify traffic is shifted to the active links and no traffic drop observed; Verify CRM that no new nexthop created
Bring back the uplink to T1	ECMP hashing/CRM	Verify traffic is now equally distributed; Verify CRM that no new nexthop created

Shutdown one BGP session to T1	ECMP hashing/CRM	Verify traffic is shifted to the active links and no traffic drop observed; Verify CRM that no new nexthop created
Bring back BGP session to T1	ECMP hashing/CRM	Verify traffic is now equally distributed; Verify CRM that no new entries created

Server Neighbor entry flushed/relearnt	Standby Forwarding	Verify no impact to tunnel traffic and no traffic fwded to neighbor directly; Verify CRM for neighbor (Implement in another PR)

Remove Loopback route	ECMP hashing	Verify traffic is equally distributed via default route
Re-add Loopback route	ECMP hashing	Verify traffic is equally distributed via loopback route

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Approach

What is the motivation for this PR?

This PR is to implement dualtor orchagent test cases.

How did you do it?

Please refer to code.

How did you verify/test it?

Verified on a dualtor testbed.

py.test --inventory ../ansible/str2,../ansible/veos --host-pattern str2-7050cx3-acs-08 --module-path ../ansible --testbed vms17-dual-t0-7050-2 --testbed_file ../ansible/testbed.csv --junit-xml=tr.xml --log-cli-level info --collect_techsupport=False --topology=t0,any,util dualtor/test_standby_tor_downstream.py
========================================================================================= test session starts =========================================================================================
collected 5 items                                                                                                                                                                                     

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
05:34:25 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
05:34:25 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
05:34:30 INFO ptfhost_utils.py:change_mac_addresses:102: Change interface MAC addresses on ptfhost 'vms17-8'
05:34:31 INFO ptfhost_utils.py:copy_ptftests_directory:62: Copy PTF test files to PTF host 'vms17-8'
05:34:40 INFO ptfhost_utils.py:run_icmp_responder:181: Start running icmp_responder
05:34:51 INFO conftest.py:generate_params_dut_hostname:744: DUTs in testbed 'vms17-dual-t0-7050-2' are: ['str2-7050cx3-acs-08', 'str2-7050cx3-acs-09']
05:34:51 INFO conftest.py:rand_one_dut_hostname:222: Randomly select dut str2-7050cx3-acs-08 for testing
05:34:51 INFO conftest.py:creds:415: dut str2-7050cx3-acs-08 belongs to groups [u'sonic', u'sonic_arista', u'str2', 'fanout']
05:34:51 INFO conftest.py:creds:427: skip empty var file ../ansible/group_vars/all/corefile_uploader.yml
05:34:51 INFO conftest.py:creds:427: skip empty var file ../ansible/group_vars/all/README.yml
05:34:56 INFO __init__.py:sanity_check:121: Prepare sanity check
05:34:56 INFO __init__.py:sanity_check:131: Found marker: m.name=topology, m.args=('t0',), m.kwargs={}
05:34:56 INFO __init__.py:sanity_check:131: Found marker: m.name=usefixtures, m.args=('apply_mock_dual_tor_tables', 'apply_mock_dual_tor_kernel_configs', 'apply_standby_state_to_orchagent', 'run_garp_service', 'run_icmp_responder'), m.kwargs={}
05:34:56 INFO __init__.py:sanity_check:157: Skip sanity check according to command line argument or configuration of test script.
05:34:56 INFO dual_tor_mock.py:set_dual_tor_state_to_orchagent:61: Applying standby state to orchagent
05:34:59 INFO ptfhost_utils.py:run_garp_service:211: Generating GARP service config file
05:35:01 INFO ptfhost_utils.py:run_garp_service:228: Starting GARP Service on PTF host
05:35:03 INFO conftest.py:generate_params_dut_hostname:744: DUTs in testbed 'vms17-dual-t0-7050-2' are: ['str2-7050cx3-acs-08', 'str2-7050cx3-acs-09']
05:35:03 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
05:35:12 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
05:35:12 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-05:35:12.log
 ^HPASSED                                                                                                                                                                                          [ 20%]
dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_t1_link_recovered 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
05:43:01 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
05:43:01 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
05:43:01 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
05:43:01 INFO conftest.py:set_crm_polling_interval:19: Setting crm polling interval to 1 seconds
05:43:03 INFO conftest.py:set_crm_polling_interval:21: Waiting 2 sec for CRM counters to become updated
05:43:07 INFO conftest.py:verify_crm_nexthop_counter_not_increased:34: Before test: crm nexthop counter = 6
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
05:43:09 INFO test_standby_tor_downstream.py:shutdown_random_one_t1_link:30: Shutting down interface PortChannel0001
05:43:51 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
05:43:51 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-05:43:51.log
 ^H ^H05:52:06 INFO test_standby_tor_downstream.py:no_shutdown_t1_link:40: Bring back interface PortChannel0001
05:52:46 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
05:52:46 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-05:52:46.log
 ^H ^HPASSED                                                                                                                                                                                          [ 40%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
06:00:47 INFO conftest.py:verify_crm_nexthop_counter_not_increased:38: Before test: crm nexthop counter = 6
06:00:47 INFO conftest.py:set_crm_polling_interval:24: Setting crm polling interval to 300 seconds

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_bgp_recovered 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
06:00:48 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
06:00:48 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
06:00:48 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
06:00:48 INFO conftest.py:set_crm_polling_interval:19: Setting crm polling interval to 1 seconds
06:00:49 INFO conftest.py:set_crm_polling_interval:21: Waiting 2 sec for CRM counters to become updated
06:00:53 INFO conftest.py:verify_crm_nexthop_counter_not_increased:34: Before test: crm nexthop counter = 6
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
06:00:55 INFO test_standby_tor_downstream.py:shutdown_random_one_bgp_session:63: Shutting down bgp session with 10.0.0.57
06:01:35 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
06:01:35 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-06:01:35.log
 ^H ^H06:09:39 INFO test_standby_tor_downstream.py:startup_bgp_session:73: Bring back bgp session with 10.0.0.57
06:10:20 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
06:10:20 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-06:10:20.log
 ^HPASSED                                                                                                                                                                                          [ 60%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
06:18:14 INFO conftest.py:verify_crm_nexthop_counter_not_increased:38: Before test: crm nexthop counter = 6
06:18:14 INFO conftest.py:set_crm_polling_interval:24: Setting crm polling interval to 300 seconds

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_removed 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
06:18:16 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
06:18:16 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
06:18:16 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
06:18:24 INFO test_standby_tor_downstream.py:remove_loopback_routes:111: Removing dual ToR peer switch loopback route
06:18:25 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
06:18:25 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-06:18:25.log
 ^H ^HPASSED                                                                                                                                                                                          [ 80%]
dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_readded 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
06:26:30 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
06:26:30 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
06:26:30 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
06:26:39 INFO test_standby_tor_downstream.py:add_loopback_routes:89: Applying dual ToR peer switch loopback route
06:26:41 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
06:26:41 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-06:26:41.log
 ^HPASSED                                                                                                                                                                                          [100%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
06:34:33 INFO dual_tor_mock.py:del_dual_tor_state_from_orchagent:97: Removing standby state from orchagent
06:34:39 INFO ptfhost_utils.py:run_icmp_responder:199: Stop running icmp_responder
 ^H06:34:40 INFO ptfhost_utils.py:copy_ptftests_directory:67: Delete PTF test files from PTF host 'vms17-8'

=============================================================================== 5 passed, 1 warnings in 3615.45 seconds ===============================================================================

Verifying on a t0 testbed.

py.test --inventory ../ansible/str2,../ansible/veos --host-pattern str2-7060cx3-acs-09 --module-path ../ansible --testbed vms17-t0-7260-2 --testbed_file ../ansible/testbed.csv --junit-xml=tr.xml --log-cli-level info --collect_techsupport=False --topology=t0,any,util dualtor/test_standby_tor_downstream.py
========================================================================================= test session starts =========================================================================================
collected 5 items                                                                                                                                                                                     

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
06:51:27 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
06:51:27 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
06:51:34 INFO ptfhost_utils.py:change_mac_addresses:102: Change interface MAC addresses on ptfhost 'vms17-9'
06:51:35 INFO ptfhost_utils.py:copy_ptftests_directory:62: Copy PTF test files to PTF host 'vms17-9'
06:51:44 INFO ptfhost_utils.py:run_icmp_responder:181: Start running icmp_responder
06:52:38 INFO conftest.py:generate_params_dut_hostname:744: DUTs in testbed 'vms17-t0-7260-2' are: ['str2-7260cx3-acs-9']
06:52:38 INFO conftest.py:rand_one_dut_hostname:222: Randomly select dut str2-7260cx3-acs-9 for testing
06:52:38 INFO conftest.py:creds:415: dut str2-7260cx3-acs-9 belongs to groups [u'sonic', u'sonic_arista64_100', u'str2', 'fanout']
06:52:38 INFO conftest.py:creds:427: skip empty var file ../ansible/group_vars/all/corefile_uploader.yml
06:52:38 INFO conftest.py:creds:427: skip empty var file ../ansible/group_vars/all/README.yml
06:52:43 INFO __init__.py:sanity_check:121: Prepare sanity check
06:52:43 INFO __init__.py:sanity_check:131: Found marker: m.name=topology, m.args=('t0',), m.kwargs={}
06:52:43 INFO __init__.py:sanity_check:131: Found marker: m.name=usefixtures, m.args=('apply_mock_dual_tor_tables', 'apply_mock_dual_tor_kernel_configs', 'apply_standby_state_to_orchagent', 'run_garp_service', 'run_icmp_responder'), m.kwargs={}
06:52:43 INFO __init__.py:sanity_check:157: Skip sanity check according to command line argument or configuration of test script.
06:53:27 INFO dual_tor_mock.py:apply_mux_cable_table_to_dut:339: Applying MUX_CABLE table
06:54:32 INFO dual_tor_mock.py:apply_tunnel_table_to_dut:310: Applying TUNNEL table
06:54:36 INFO dual_tor_mock.py:apply_peer_switch_table_to_dut:287: Applying PEER_SWITCH table
06:54:39 INFO dual_tor_mock.py:apply_mock_dual_tor_tables:406: Done applying database tables for dual ToR mock
06:54:39 INFO dual_tor_mock.py:apply_dual_tor_peer_switch_route:254: Applying dual ToR peer switch loopback route
06:54:51 INFO dual_tor_mock.py:apply_dual_tor_neigh_entries:226: Applying dual ToR neighbor entries
06:54:53 INFO dual_tor_mock.py:apply_mock_dual_tor_kernel_configs:417: Done applying kernel configs for dual ToR mock
06:54:53 INFO dual_tor_mock.py:set_dual_tor_state_to_orchagent:62: Applying standby state to orchagent
06:54:57 INFO ptfhost_utils.py:run_garp_service:211: Generating GARP service config file
06:54:59 INFO ptfhost_utils.py:run_garp_service:228: Starting GARP Service on PTF host
06:55:01 INFO conftest.py:generate_params_dut_hostname:744: DUTs in testbed 'vms17-t0-7260-2' are: ['str2-7260cx3-acs-9']
06:55:01 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
06:55:52 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
06:55:52 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-06:55:52.log
 ^H ^HPASSED                                                                                                                                                                                          [ 20%]
dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_t1_link_recovered 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
07:03:08 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
07:03:08 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
07:03:08 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
07:03:08 INFO conftest.py:set_crm_polling_interval:19: Setting crm polling interval to 1 seconds
07:03:14 INFO conftest.py:set_crm_polling_interval:21: Waiting 2 sec for CRM counters to become updated
07:03:19 INFO conftest.py:verify_crm_nexthop_counter_not_increased:34: Before test: crm nexthop counter = 5
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
07:03:20 INFO test_standby_tor_downstream.py:shutdown_random_one_t1_link:30: Shutting down interface PortChannel0002
07:04:43 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
07:04:43 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-07:04:43.log
 ^H ^H07:11:42 INFO test_standby_tor_downstream.py:no_shutdown_t1_link:40: Bring back interface PortChannel0002
07:13:09 INFO test_standby_tor_downstream.py:remove_loopback_routes:111: Removing dual ToR peer switch loopback route
07:13:10 INFO test_standby_tor_downstream.py:add_loopback_routes:89: Applying dual ToR peer switch loopback route
07:13:12 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
07:13:12 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-07:13:12.log
 ^HPASSED                                                                                                                                                                                          [ 40%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
07:20:19 INFO conftest.py:verify_crm_nexthop_counter_not_increased:38: Before test: crm nexthop counter = 5
07:20:19 INFO conftest.py:set_crm_polling_interval:24: Setting crm polling interval to 300 seconds

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_bgp_recovered 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
07:20:20 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
07:20:20 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
07:20:20 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
07:20:20 INFO conftest.py:set_crm_polling_interval:19: Setting crm polling interval to 1 seconds
07:20:21 INFO conftest.py:set_crm_polling_interval:21: Waiting 2 sec for CRM counters to become updated
07:20:26 INFO conftest.py:verify_crm_nexthop_counter_not_increased:34: Before test: crm nexthop counter = 5
-------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------
07:20:28 INFO test_standby_tor_downstream.py:shutdown_random_one_bgp_session:63: Shutting down bgp session with 10.0.0.35
 ^H07:21:52 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
07:21:52 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-07:21:52.log
 ^H07:29:07 INFO test_standby_tor_downstream.py:startup_bgp_session:73: Bring back bgp session with 10.0.0.35
07:30:33 INFO dual_tor_utils.py:check_tunnel_balance:622: run ptf test for verifying IPinIP tunnel balance
07:30:33 INFO dual_tor_utils.py:check_tunnel_balance:625: PTF log file: /tmp/ip_in_ip_tunnel_test.2021-04-06-07:30:33.log
 ^H ^HPASSED                                                                                                                                                                                          [ 60%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
07:37:58 INFO conftest.py:verify_crm_nexthop_counter_not_increased:38: Before test: crm nexthop counter = 5
07:37:58 INFO conftest.py:set_crm_polling_interval:24: Setting crm polling interval to 300 seconds

dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_removed 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
07:37:59 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
07:37:59 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
07:37:59 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
SKIPPED                                                                                                                                                                                         [ 80%]
dualtor/test_standby_tor_downstream.py::test_standby_tor_downstream_loopback_route_readded 
------------------------------------------------------------------------------------------- live log setup --------------------------------------------------------------------------------------------
07:37:59 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
07:37:59 INFO __init__.py:check_test_completeness:139: Test has no defined levels. Continue without test completeness checks
07:37:59 INFO __init__.py:loganalyzer:17: Log analyzer is disabled
SKIPPED                                                                                                                                                                                         [100%]
------------------------------------------------------------------------------------------ live log teardown ------------------------------------------------------------------------------------------
07:38:04 INFO dual_tor_mock.py:del_dual_tor_state_from_orchagent:98: Removing standby state from orchagent
07:38:06 INFO dual_tor_mock.py:apply_dual_tor_neigh_entries:241: Removing dual ToR neighbor entries
07:38:07 INFO dual_tor_mock.py:apply_dual_tor_peer_switch_route:276: Removing dual ToR peer switch loopback route
07:38:08 INFO dual_tor_mock.py:apply_peer_switch_table_to_dut:298: Removing peer switch table
07:38:11 INFO dual_tor_mock.py:apply_tunnel_table_to_dut:329: Removing tunnel table
07:38:11 INFO dual_tor_mock.py:apply_mux_cable_table_to_dut:358: Removing mux cable table
07:39:15 INFO ptfhost_utils.py:run_icmp_responder:199: Stop running icmp_responder
07:39:20 INFO ptfhost_utils.py:copy_ptftests_directory:67: Delete PTF test files from PTF host 'vms17-9'

-------------------------------------------------------------------------- generated xml file: /data/sonic-mgmt/tests/tr.xml --------------------------------------------------------------------------
========================================================================= 3 passed, 2 skipped, 1 warnings in 2873.49 seconds ==========================================================================

Any platform specific information?

No.

Supported testbed topology if it's a new test case?

No.

Documentation

lgtm-com · 2021-03-09T08:25:56Z

This pull request introduces 2 alerts and fixes 1 when merging 26bef205c7317b6d4db61146bb62c94c11b95f74 into b4b6d21 - view on LGTM.com

new alerts:

2 for Module-level cyclic import

fixed alerts:

1 for Unused import

lgtm-com · 2021-03-09T11:44:44Z

This pull request fixes 1 alert when merging 80494d91a65a824eaaa6c1e9b81f3365c325e0df into b4b6d21 - view on LGTM.com

fixed alerts:

1 for Unused import

lgtm-com · 2021-03-16T09:26:12Z

This pull request fixes 1 alert when merging 641537b23fd25fb4e1be7f11bfe16037ad5bffd4 into eb75a74 - view on LGTM.com

fixed alerts:

1 for Unused import

lgtm-com · 2021-03-17T05:19:43Z

This pull request fixes 1 alert when merging 95098cfcc439c984e0228f6e660ef92063b58ef0 into ae8b02a - view on LGTM.com

fixed alerts:

1 for Unused import

tests/common/dualtor/dual_tor_utils.py

tests/dualtor/test_standby_tor_downstream.py

Signed-off-by: bingwang <[email protected]>

theasianpianist · 2021-04-01T18:27:29Z

Overall looks good. Please include some test output in the PR description, and make the test filename more specific. Thanks!

theasianpianist · 2021-04-05T22:26:23Z

tests/dualtor/test_standby_tor_downstream.py

+    check_tunnel_balance(**params)
+
+
+def test_standby_tor_downstream_loopback_route_removed(ptfhost, rand_selected_dut, rand_unselected_dut, tbinfo):


Please combine the loopback test cases into one case (similar to the other cases above)

Updated. Thanks

Signed-off-by: bingwang <[email protected]>

lgtm-com · 2021-04-06T07:41:24Z

This pull request fixes 1 alert when merging d93dffa into 18dce2d - view on LGTM.com

fixed alerts:

1 for Unused import

Signed-off-by: bingwang <[email protected]>

…mgmt into orchagent_crm

lgtm-com · 2021-04-06T08:58:07Z

This pull request fixes 1 alert when merging 0058c43 into 18dce2d - view on LGTM.com

fixed alerts:

1 for Unused import

gechiang · 2021-04-06T15:21:23Z

@bingwang-ms Does your testcase cover TTL/DSCP (pipe/uniform mode) validation for both encap and decap cases?
This will help shake out SAI issues.
Thanks!

theasianpianist · 2021-04-07T01:41:29Z

tests/common/dualtor/dual_tor_mock.py

+
+
+@pytest.fixture(scope='module')
+def apply_dualtor_subtype_to_th2(duthosts):


This field already gets added in apply_peer_switch_table_to_dut, is it possible to just add the SWSS restart functionality to that fixture in case of TH2 ASIC?

Thanks. I didn't realise the subtype has been added in apply_peer_switch_table_to_dut.
@gechiang Could you help to confirm if it's no hurt to add subtype: DualToR for all platform, and does dualToR equal to DualToR? Thanks

@bingwang-ms the entry in apply_peer_switch_table_to_dut is supposed to be DualToR, looks like I made a typo when I first wrote it 😁 would appreciate if you could fix that for me lol

I believe the subtype field is needed by orchagent for all platforms, @prsunny can you confirm?

@bingwang-ms
The subtype field if you are matching with case sensitive then it needs to match "DualToR".
I did a quick check on what may be impacted if subtype: "DualToR" is added blindly:

The following check for DUALTOR and perform Stopping MUX contaniner:
./src/sonic-utilities/scripts/reboot:SUBTYPE=$(sonic-cfggen -d -v DEVICE_METADATA.localhost.subtype)

The following during config Reload ARP/FDB entries are cached if DualToR:
./src/sonic-utilities/config/main.py: cache_arp_table = not disable_arp_cache and 'subtype' in localhost_metadata and localhost_metadata['subtype'].lower() == 'dualtor'

The following check for DualToR and Mux container is optionally enabled:
../files/build_templates/init_cfg.json.j2:{%- if include_mux == "y" %}{% do features.append(("mux", "{% if 'subtype' in DEVICE_METADATA['localhost'] and DEVICE_METADATA['localhost']['subtype'] == 'DualToR' %}enabled{% else %}always_disabled{% endif %}", false, "enabled")) %}{% endif %}

When DHCP docker started, if DualToR is configured, it enable some special options:
./dockers/docker-dhcp-relay/docker-dhcp-relay.supervisord.conf.j2:{% if 'subtype' in DEVICE_METADATA['localhost'] and DEVICE_METADATA['localhost']['subtype'] == 'DualToR' %} -U Loopback0 -dt{% endif -%}

So if your testcase is not involve in any of the above and you always restore it back (remove the subtype setting if it was set by your testcase only) then it is ok to set it for all platforms.

@bingwang-ms the entry in apply_peer_switch_table_to_dut is supposed to be DualToR, looks like I made a typo when I first wrote it 😁 would appreciate if you could fix that for me lol

I believe the subtype field is needed by orchagent for all platforms, @prsunny can you confirm?

Sure. I will corretc it.

Updated. Thanks

bingwang-ms · 2021-04-07T08:05:02Z

@bingwang-ms Does your testcase cover TTL/DSCP (pipe/uniform mode) validation for both encap and decap cases?
This will help shake out SAI issues.
Thanks!

Yes. the TTL/DSCP validation is covered by another case implemented in https://github.com/Azure/sonic-mgmt/blob/master/tests/dualtor/test_ipinip.py

Signed-off-by: bingwang <[email protected]>

lgtm-com · 2021-04-08T08:13:11Z

This pull request fixes 1 alert when merging d714fac into d63b340 - view on LGTM.com

fixed alerts:

1 for Unused import

theasianpianist · 2021-04-08T20:49:14Z

tests/dualtor/test_standby_tor_downstream.py

+    """
+    Verify traffic is equally distributed via loopback route
+    """
+    pt_require('dualtor' in tbinfo['topo']['name'], "Only run on dualtor testbed")


We want to be able to run this test on single ToR testbeds as well, what was the reasoning to mark this part as dual ToR only?

Thanks for the good catch. It should be test_standby_tor_downstream_loopback_route_readded. I deleted the function defination in recent merge by mistake😁. I think test_standby_tor_downstream_loopback_route_readded should run only on dualtor testbed.

@prsunny can you confirm this case is dualtor only? I was under the impression all the orchagent cases were able to run on single ToR.

Signed-off-by: bingwang <[email protected]>

lgtm-com · 2021-04-09T01:58:34Z

This pull request fixes 1 alert when merging 57116a2 into 99b0bae - view on LGTM.com

fixed alerts:

1 for Unused import

theasianpianist · 2021-04-09T17:43:03Z

Also could you change the filename to test_orchagent_standby_tor_downstream or something more specific? Thanks!

Signed-off-by: bingwang <[email protected]>

lgtm-com · 2021-04-12T06:09:54Z

This pull request fixes 1 alert when merging 413bfe0 into 84f004f - view on LGTM.com

fixed alerts:

1 for Unused import

theasianpianist

thanks!

…onic-net#3110) * Implement dualtor T1 -> Standby Tor test cases. Signed-off-by: bingwang <[email protected]>

bingwang-ms requested a review from a team as a code owner March 9, 2021 07:50

This was linked to issues Mar 19, 2021

BGP Failure Test Case #2806

Closed

T1 Link Failure Test Case #2807

Closed

bingwang-ms requested a review from theasianpianist March 26, 2021 01:50

theasianpianist reviewed Mar 29, 2021

View reviewed changes

tests/common/dualtor/dual_tor_utils.py Show resolved Hide resolved

theasianpianist reviewed Mar 29, 2021

View reviewed changes

tests/dualtor/test_standby_tor_downstream.py Outdated Show resolved Hide resolved

theasianpianist reviewed Mar 29, 2021

View reviewed changes

tests/dualtor/test_standby_tor_downstream.py Outdated Show resolved Hide resolved

theasianpianist reviewed Mar 29, 2021

View reviewed changes

tests/dualtor/test_standby_tor_downstream.py Outdated Show resolved Hide resolved

bingwang-ms added 5 commits March 31, 2021 20:48

Implement dualtor T1 -> Standby Tor test cases.

f8b3597

Signed-off-by: bingwang <[email protected]>

Fix LGTM

28616c0

Signed-off-by: bingwang <[email protected]>

Fix bug for running on mocked dualtor testbed

463f363

Signed-off-by: bingwang <[email protected]>

Fix mocked dualtor.

144e55c

Signed-off-by: bingwang <[email protected]>

Fix comment

1e45a3a

Signed-off-by: bingwang <[email protected]>

theasianpianist reviewed Apr 5, 2021

View reviewed changes

theasianpianist requested a review from prsunny April 5, 2021 23:26

Add subtype for th2

fe621f4

Signed-off-by: bingwang <[email protected]>

bingwang-ms force-pushed the orchagent_crm branch from 1f3f20e to fe621f4 Compare April 6, 2021 07:02

Merge branch 'master' into orchagent_crm

d93dffa

bingwang-ms added 2 commits April 6, 2021 01:25

Fix comment

6d3555d

Signed-off-by: bingwang <[email protected]>

Merge branch 'orchagent_crm' of https://github.com/bingwang-ms/sonic-…

0058c43

…mgmt into orchagent_crm

bingwang-ms mentioned this pull request Apr 6, 2021

IPinIP test case support for Standalone TH2 (7260CX3) to modify config DB DEVICE_METADATA localhost subtype #3256

Closed

bingwang-ms requested a review from gechiang April 6, 2021 08:30

theasianpianist reviewed Apr 7, 2021

View reviewed changes

bingwang-ms and others added 2 commits April 7, 2021 23:01

Update subtype in apply_peer_switch_table_to_dut

34500dd

Signed-off-by: bingwang <[email protected]>

Merge branch 'master' into orchagent_crm

d714fac

theasianpianist reviewed Apr 8, 2021

View reviewed changes

Fix unexpectedly deleted line

57116a2

Signed-off-by: bingwang <[email protected]>

Rename file

413bfe0

Signed-off-by: bingwang <[email protected]>

bingwang-ms force-pushed the orchagent_crm branch from da539b8 to 413bfe0 Compare April 12, 2021 05:40

theasianpianist approved these changes Apr 12, 2021

View reviewed changes

bingwang-ms merged commit cb401ec into sonic-net:master Apr 13, 2021

saravanansv pushed a commit to saravanansv/sonic-mgmt that referenced this pull request May 6, 2021

[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. (s…

aa6a844

…onic-net#3110) * Implement dualtor T1 -> Standby Tor test cases. Signed-off-by: bingwang <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. #3110

[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. #3110

bingwang-ms commented Mar 9, 2021 •

edited

Loading

lgtm-com bot commented Mar 9, 2021

lgtm-com bot commented Mar 9, 2021

lgtm-com bot commented Mar 16, 2021

lgtm-com bot commented Mar 17, 2021

theasianpianist commented Apr 1, 2021

theasianpianist Apr 5, 2021

bingwang-ms Apr 6, 2021

lgtm-com bot commented Apr 6, 2021

lgtm-com bot commented Apr 6, 2021

gechiang commented Apr 6, 2021

theasianpianist Apr 7, 2021

bingwang-ms Apr 7, 2021

theasianpianist Apr 7, 2021

gechiang Apr 7, 2021

bingwang-ms Apr 8, 2021

bingwang-ms Apr 8, 2021

bingwang-ms commented Apr 7, 2021

lgtm-com bot commented Apr 8, 2021

theasianpianist Apr 8, 2021

bingwang-ms Apr 9, 2021

theasianpianist Apr 9, 2021

lgtm-com bot commented Apr 9, 2021

theasianpianist commented Apr 9, 2021

lgtm-com bot commented Apr 12, 2021

theasianpianist left a comment

		check_tunnel_balance(**params)


		def test_standby_tor_downstream_loopback_route_removed(ptfhost, rand_selected_dut, rand_unselected_dut, tbinfo):



		@pytest.fixture(scope='module')
		def apply_dualtor_subtype_to_th2(duthosts):

[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. #3110

[dualtor] Implement dualtor T1 -> Standby Tor orchagent test cases. #3110

Conversation

bingwang-ms commented Mar 9, 2021 • edited Loading

Description of PR

Type of change

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

lgtm-com bot commented Mar 9, 2021

lgtm-com bot commented Mar 9, 2021

lgtm-com bot commented Mar 16, 2021

lgtm-com bot commented Mar 17, 2021

theasianpianist commented Apr 1, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgtm-com bot commented Apr 6, 2021

lgtm-com bot commented Apr 6, 2021

gechiang commented Apr 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bingwang-ms commented Apr 7, 2021

lgtm-com bot commented Apr 8, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgtm-com bot commented Apr 9, 2021

theasianpianist commented Apr 9, 2021

lgtm-com bot commented Apr 12, 2021

theasianpianist left a comment

Choose a reason for hiding this comment

bingwang-ms commented Mar 9, 2021 •

edited

Loading