Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

swss docker warm restart support #1

Closed
wants to merge 112 commits into from
Closed

swss docker warm restart support #1

wants to merge 112 commits into from

Conversation

jipanyang
Copy link
Owner

@jipanyang jipanyang commented Jun 6, 2018 •
edited
Loading

What I did
swss state restore,
port state sync up.
FDB sync up.

Why I did it

How I verified it
Added vs test cases for swss restore, port state sync up, fdb sync up and crm check

test_warm_reboot.py::test_swss_warm_restore PASSED                                                                                                         [ 95%]
test_warm_reboot.py::test_swss_port_state_syncup PASSED                                                                                                    [ 97%]
test_warm_reboot.py::test_swss_fdb_syncup_and_crm PASSED  

jipan@sonic-build:~/igbpatch/vs/sonic-buildimage/src/sonic-swss/tests$ sudo pytest -v --dvsname=vs --notempview 
[sudo] password for jipan: 
====================================================================== test session starts =======================================================================
platform linux2 -- Python 2.7.12, pytest-3.3.0, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .cache
rootdir: /home/jipan/igbpatch/vs/sonic-buildimage/src/sonic-swss/tests, inifile:
collected 45 items                                                                                                                                               

test_acl.py::TestAcl::test_AclTableCreation PASSED                                                                                                         [  2%]
test_acl.py::TestAcl::test_AclRuleL4SrcPort PASSED                                                                                                         [  4%]
test_acl.py::TestAcl::test_AclTableDeletion PASSED                                                                                                         [  6%]
test_acl.py::TestAcl::test_V6AclTableCreation PASSED                                                                                                       [  8%]
test_acl.py::TestAcl::test_V6AclRuleIPv6Any PASSED                                                                                                         [ 11%]
test_acl.py::TestAcl::test_V6AclRuleIPv6AnyDrop PASSED                                                                                                     [ 13%]
test_acl.py::TestAcl::test_V6AclRuleIpProtocol PASSED                                                                                                      [ 15%]
test_acl.py::TestAcl::test_V6AclRuleSrcIPv6 PASSED                                                                                                         [ 17%]
test_acl.py::TestAcl::test_V6AclRuleDstIPv6 PASSED                                                                                                         [ 20%]
test_acl.py::TestAcl::test_V6AclRuleL4SrcPort PASSED                                                                                                       [ 22%]
test_acl.py::TestAcl::test_V6AclRuleL4DstPort PASSED                                                                                                       [ 24%]
test_acl.py::TestAcl::test_V6AclRuleTCPFlags PASSED                                                                                                        [ 26%]
test_acl.py::TestAcl::test_V6AclRuleL4SrcPortRange PASSED                                                                                                  [ 28%]
test_acl.py::TestAcl::test_V6AclRuleL4DstPortRange PASSED                                                                                                  [ 31%]
test_acl.py::TestAcl::test_V6AclTableDeletion PASSED                                                                                                       [ 33%]
test_acl.py::TestAcl::test_InsertAclRuleBetweenPriorities PASSED                                                                                           [ 35%]
test_acl.py::TestAcl::test_AclTableCreationOnLAGMember PASSED                                                                                              [ 37%]
test_acl.py::TestAcl::test_AclTableCreationOnLAG PASSED                                                                                                    [ 40%]
test_acl.py::TestAcl::test_AclTableCreationBeforeLAG PASSED                                                                                                [ 42%]
test_crm.py::test_CrmFdbEntry PASSED                                                                                                                       [ 44%]
test_crm.py::test_CrmIpv4Route PASSED                                                                                                                      [ 46%]
test_crm.py::test_CrmIpv6Route PASSED                                                                                                                      [ 48%]
test_crm.py::test_CrmIpv4Nexthop PASSED                                                                                                                    [ 51%]
test_crm.py::test_CrmIpv6Nexthop PASSED                                                                                                                    [ 53%]
test_crm.py::test_CrmIpv4Neighbor PASSED                                                                                                                   [ 55%]
test_crm.py::test_CrmIpv6Neighbor PASSED                                                                                                                   [ 57%]
test_crm.py::test_CrmNexthopGroup PASSED                                                                                                                   [ 60%]
test_crm.py::test_CrmNexthopGroupMember PASSED                                                                                                             [ 62%]
test_crm.py::test_CrmAcl PASSED                                                                                                                            [ 64%]
test_dirbcast.py::test_DirectedBroadcast PASSED                                                                                                            [ 66%]
test_fdb.py::test_FDBAddedAfterMemberCreated PASSED                                                                                                        [ 68%]
test_interface.py::test_InterfaceIpChange PASSED                                                                                                           [ 71%]
test_nhg.py::test_route_nhg PASSED                                                                                                                         [ 73%]
test_port.py::test_PortNotification PASSED                                                                                                                 [ 75%]
test_portchannel.py::test_PortChannel PASSED                                                                                                               [ 77%]
test_route.py::test_RouteAdd PASSED                                                                                                                        [ 80%]
test_setro.py::test_SetReadOnlyAttribute PASSED                                                                                                            [ 82%]
test_speed.py::TestSpeedSet::test_SpeedAndBufferSet PASSED                                                                                                 [ 84%]
test_vlan.py::test_VlanMemberCreation PASSED                                                                                                               [ 86%]
test_vrf.py::test_VRFOrch_Comprehensive PASSED                                                                                                             [ 88%]
test_vrf.py::test_VRFOrch PASSED                                                                                                                           [ 91%]
test_vrf.py::test_VRFOrch_Update PASSED                                                                                                                    [ 93%]
test_warm_reboot.py::test_swss_warm_restore PASSED                                                                                                         [ 95%]
test_warm_reboot.py::test_swss_port_state_syncup PASSED                                                                                                    [ 97%]
test_warm_reboot.py::test_swss_fdb_syncup_and_crm PASSED                                                                                                   [100%]

================================================================== 45 passed in 630.11 seconds ===================================================================
jipan@sonic-build:~/igbpatch/vs/sonic-buildimage/src/sonic-swss/tests$

Details if related

simone-dell and others added 2 commits June 8, 2018 15:09
@simone-dell @lguohan
[acl]: test different subnet masks (sonic-net#481)
44309ef 
* Add files via upload

* Test case: ACL rule with diff subnet masks

Added a test case with ACL rules with different mask /8 /19 ,etc.  Verified the proper subnet masks were reflected in the ASICDB.

* changing addition of acl rules to a loop to avoid redundancy

* added a helper function to improve code efficiency

* Update test_acl.py
@pavel-shirshov
[portsorch]: Bring the physical ports up are only after buffer config…
2176688 
…uration was applied (sonic-net#515)

* Don't up ports, until buffer configuration is applied

* Set MTU first, then set port state to UP

* Introduce the test

* Use logical operator && for boolean values
@jipanyang
Copy link
Owner Author

First swss warm restore test case:
sudo pytest -v --dvsname=vs --notempview test_warm_reboot.py

jipanyang and others added 12 commits June 13, 2018 02:05
@jipanyang @lguohan
VS: Check oper_status of the link being brought up instead of Etherne…
ad61242 
…t0 (sonic-net#522)

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@prsunny
Allocate buffer for 256 entries for ACL resource query (sonic-net#523)
6189eaa 
* Allocate buffer for 256 entries for ACL resource query

* Changed to MACRO
@sihuihan88 @pavel-shirshov
[counter]: add support to enable/disable counter query (sonic-net#511)
98ad0bd 
Signed-off-by: Sihui Han <sihan@microsoft.com>
@pavel-shirshov @lguohan
Fix double freeing bug in fdb event handler (sonic-net#525)
27b29e3
@sihuihan88
[counterpoll]:enable/disable pfcwd counter query (sonic-net#526)
72a7f3a 
Signed-off-by: Sihui Han <sihan@microsoft.com>
@pavel-shirshov
Pospone QueueMap initialization until activation of counters (sonic-n…
258ffe6 
…et#527)
@pavel-shirshov
Initialize CounterCheckOrch with port information after QueueMaps ini…
90ea7a4 
…tialization (sonic-net#528)

* Pospone QueueMap initialization until activation of counters

* Initialize CounterCheckOrch after we initialized the QueueMaps
@qiluo-msft
[doc] Fix the build checker links (sonic-net#531)
19e6c72
@pavel-shirshov
Generate queue maps only for front panel ports (sonic-net#532)
82a95a1 
* Generate queue maps only for front panel ports
@qiluo-msft @lguohan
Update README.md (sonic-net#533)
23bb493
@pavel-shirshov @lguohan
Check m_port_ready_list_ref value for presense before using the value (…
22bb2cd 
…sonic-net#530)

* Pospone QueueMap initialization until activation of counters

* Check that port ready list has the information before referencing the information
@prsunny @lguohan
SAI 1.3 header changes (sonic-net#520)
19f9108
@pavel-shirshov @lguohan
Create empty ready lists for buffers by default (sonic-net#535)
5d31b2d 
* Pospone QueueMap initialization until activation of counters

* Generate queue maps only for front panel ports

* Create empty buffer lists by default
@jipanyang jipanyang force-pushed the idempotent branch 2 times, most recently from 163bc1a to 0a96942 Compare July 11, 2018 03:28
keboliu and others added 9 commits July 11, 2018 11:43
@keboliu @lguohan
[Aclorch] fix Acl bind point type issue (sonic-net#516)
37b1001 
*  [aclorch] fix acl bind point type issue

* add support for vlan bind point type support

* Revert "add support for vlan bind point type support"

This reverts commit e026cc1.
Update tests/README.md
7a2c61d 
Add the note when encountering the client/server version mismatch issue.
@lguohan
Add support for AN and FEC to be specified in port_config.ini (sonic-…
762e7da 
…net#534)

* Add support for AN and FEC to be specified in port_config.ini

* do not set autoneg if it is already set

* set port adv speed for AN and add test

Signed-off-by: Guohan Lu <gulv@microsoft.com>
@lguohan
[MirrorOrch]: Fix bug when NULL is returned from the getenv function (s…
b976487 
…onic-net#537)

This fixes the error:
Failed due to exception: basic_string::_S_construct null not valid

when the environment variable 'platform' is not set.

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
@lguohan
[aclorch]: Add support for value/mask match for DSCP value (sonic-net…
620fe95 
…#536)

Add the unit test test_mirror.py which currently covers:
- ACL mirror table creation
- mirror session activation
- ACL rule DSCP with/without mask
- ACL mirror table removal

Signed-off-by: Shu0T1an ChenG <shuche@microsoft.com>
@shruthi9 @lguohan
swss: Dataplane telemetry support in SONiC (sonic-net#521)
dcf6c90 
* initial barefoot support october 2017

* import changes from telemetry branch

* Fix merge issues w.r.t rel_6_0 branch

* add validate and get port speed APIs - cleanup deletes from earlier checkin

* missed changes from earlier

* missed integration delete

* trying to get close to master version - and not break other vendor

* set the port config in the ASIC_DB

* update hostif oper status besides the DB update

* Fix compilation issues

* merge closer to master - including spaces and comments

* fix typos
enable mirror

* cosmetic fix

* Fix errors due to saiacl.h changes

* Fix more errors due to sai header file changes

* fix the order of nexthop/neighbor delete

* Fix compilation issues and few issues seen in testing

* Changes needed to add new Dtel api support to sairedis

* Report session DTel table related fixes

* Add more error handling code

* Fix logic to read table name and keys from m_toSync map

* Update value for dtel actions

* Handle boolean attributes to accept true/false as well as presence/absence

* Fix merge errors

* Incorporate configdb related changes for dtel

* fix code merge issue

* Incorporate configdb related changes for dtel

* add pfc_detect lua script

* Changes to incorporate new DTel SAI changes

* SONiC changes due to DTel experimental SAI changes

* cleanup configure.ac to allow barefoot platform includes

* fix typo

* revert commented tunneldecap

* test selective tunneldecap

* revert too ambitious an attempt and just push_back into vector!

* change fec mode to string (from integer)

* cleanup based on review

* closer to 201712 - cosmetic cleanup

* address review comments

* fix format

* Support for platforms based on Barefoot Networks' device (sonic-net#452)

* Fix issue with "config save" followed by reboot

* Temp remove crmorch

Currently orchagent crashes if crm is running. Needs to be fixed

* Initial code changes to address community design review comments

* Temp remove crmorch

Currently orchagent crashes if crm is running. Needs to be fixed
P.S This is actual commit that removes crm. Previous commit was to correct a typo, and has wrong commit message

* More changes to address review comments

* Fix build errors

* Fix for orchagent crash

* Fix queue report deletion error

* Bug fixes

* Re-enable crm orchagent (sonic-net#3)

* Add support for new watchlist attribute to enable/disable tail drop reporting

* Add new VS test for Dataplane Telemetry feature

* Add support for AN and FEC to be specified in port_config.ini

* do not set autoneg if it is already set

* Address review comments

* Fix compilation errors

* set port adv speed for AN and add test

Signed-off-by: Guohan Lu <gulv@microsoft.com>

* Convert all the tabs to spaces

* Bring in changes for port an/fec from azure master

* support autoneg and fec in port config ini file
* if autoneg is specified, along with speed; speed is set seperately as port attribute

* Address review comments

* Remove trailing whitespaces

* Fix vs test indentation

* Address review comments

* Fix for VS test

* Fix VS test failure

* Fix for VS test

* More VS test fixes

* Fix for test_mirror VS test that was failing due to new DTel ACL tables
@jipanyang
Set objec owner to distinguish the objects which have the same attrib…
b2740e6 
…utes.

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Data restore for warm restart
9e65e3b
@jipanyang
Add warm start check via DB
c738366
lguohan and others added 6 commits August 3, 2018 16:06
@lguohan
fix intermitent failures in test_crm (sonic-net#563)
1fdcec6 
add acl table created by dtel to acl default tables
@sihuihan88 @lguohan
[aclorch]: only bind to port for ACL_TABLE_PFCWD type (sonic-net#561)
6556b26 
* [aclorch]: only bind to port for ACL_TABLE_PFCWD type

Signed-off-by: Sihui Han <sihan@microsoft.com>

* make the code easier to read

Signed-off-by: Sihui Han <sihan@microsoft.com>
@jipanyang
Fall back to cold start if asked to do warm start but data not available
ae4c79b 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Use the new Table::getEntry() and Table::setEntry to replace redisCli…
2c50a8f 
…ent operations

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Use the new Table:hget() and Table:hset() api
0cda262 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Add illustration about warm start knob usage
95c53e4 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
jipanyang and others added 9 commits August 6, 2018 15:52
@jipanyang
Update warm restart timer schema
88fc696 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@qiluo-msft
Revert "Port table should be ProducerTable" (sonic-net#564)
b1cf199
@qiluo-msft
Improve PortsOrch::doPortTask() (sonic-net#566)
0dd1769 
* Simplify PortsOrch::doPortTask
* Revert "Simplify PortsOrch::doPortTask", but keep unit test
* Fix handling m_autoneg, speed, mtu, admin_status, fec_mode before PortConfigDone
* Refine unit test with swsscommon.ProducerStateTable
@jipanyang @lguohan
Warm reboot: Add basic schema for warm start in configDB and stateDB. (…
674a5e6 
…sonic-net#553)

* Add basic schema for warm start schema in configDB and application DB.

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Move warm start table for process stats to state DB

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Add reconciliation timer entry in configDB warm restart table.

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Update warm restart timer schema

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* There might be more than one timer at system level or individual docker

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@qiluo-msft
Warm reboot for PortsOrch (sonic-net#551)
bfce363 
* Fix addExistingData consumer converstion
* Add more addExistingData()
* Warm reboot for PortsOrch
* Remove calling doPortConfigDoneTask in ctor
* Remove unused function signature
@jipanyang
Merge remote-tracking branch 'origin/master' into idempotent
53f7803
@jipanyang
Test script update to accomdate the new portorch warm start implement…
5a114a3 
…ation

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang @lguohan
Warm reboot: Add common warm start functions to be used by all SWSS p…
8779018 
…rocesses (sonic-net#547)

* Add common warm start functions to be used by all SWSS processes

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Use updated state schema

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Adapt to the new warm reboot schema

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Use the new Table::getEntry() and Table::setEntry to replace redisClient operations

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* use the new Table:hget() and Table:hset() APIs

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>

* Add illustration about warm start knob usage

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Set Executor name as constructor phase
2bee552 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
jipanyang and others added 12 commits August 9, 2018 16:08
@jipanyang
Use common Consumer::dumpTuple() for Orch::recordTuple() and Orch::du…
39a6094 
…mpToSyncTasks()

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Remove unneccessary netlink dump for ports
a2c778d 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Use Table::hget() to simplify the oper_status retrieval processing.
0d97ae1 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@stepanblyschak @lguohan
Add test for IPv4/IPv6 tunnels creation (sonic-net#552)
d220a80 
Signed-off-by: stepanb <stepanb@mellanox.com>
@jipanyang
Add more comment for port state sync up
baa12a0 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Simplify FDB sync up processing
7a2c68c 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Remove unused header include
9fe5a87 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Use appDB for warm restart check notification channel
b574cb6 
Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@jipanyang
Change the validation of pending task after restore to NOTICE level f…
da2e02a 
…or now.

Signed-off-by: Jipan Yang <jipan.yang@alibaba-inc.com>
@qiluo-msft
Warm reboot for general Orch (sonic-net#572)
c429ddb 
* Warm reboot for BufferOrch
* Add log for warm reboot
* Add bake() interface
* OrchDaemon supports warm start
@dzhangalibaba @qiluo-msft
swss: flush g_asicState after each event is done (sonic-net#570)
fa7008f 
* swss: flush g_asicState after each event is done
* add flush() after event is handled in case some entries are still in buffer, don't wait
* with the changes in sairedis and swss-common, route performance improved by 200~300 routes/sec
* swss-common: remove unnecessary flush() in timeout case and update comment
* remove unnecessary flush() in timeout case and update comment
@jipanyang
Merge remote-tracking branch 'origin/master' into idempotent
7637a36
@jipanyang jipanyang closed this Aug 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.