Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Mellanox] Adjust thermal control policies #8

Closed
wants to merge 26 commits into from
Closed

[Mellanox] Adjust thermal control policies #8

wants to merge 26 commits into from

Conversation

Junchao-Mellanox
Copy link
Owner

- What I did

  1. Enable kernel thermal algorithm by default
  2. Add a policy to detect FAN broken, if any FAN broken, set FAN speed to 100%
  3. If FAN/PSU back to normal, enable kernel thermal algorithm and count on it to adjust the FAN speed
  4. Add a few unit test cases for new code

- How I did it

  1. Change default thermal_policy.json
  2. Add a new condition to detect FAN broken
  3. Correct start/stop thermal algorithm logic

- How to verify it

Manual test and regression test

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Junchao-Mellanox and others added 20 commits March 11, 2020 20:39
@Junchao-Mellanox
[thermal control] Fix pmon docker stop issue on 3800
45f1229 
Conflicts:
	device/mellanox/x86_64-mlnx_msn2700-r0/plugins/sfputil.py
	platform/mellanox/mlnx-platform-api/sonic_platform/thermal.py
@Junchao-Mellanox
[thermal fix] Fix QA test issue
23bf172 
Conflicts:
	platform/mellanox/mlnx-platform-api/sonic_platform/psu.py
@stephenxs @Junchao-Mellanox
Fix thermal control issues
a1aaa93 
Conflicts:
	platform/mellanox/mlnx-platform-api/sonic_platform/psu.py
@Junchao-Mellanox
[thermal fix] change psu._get_power_available_status to psu.get_power…
7f341f9 
…_available_status
@Junchao-Mellanox
[thermal fix] adjust log for PSU absence and power absence
3206e95
@Junchao-Mellanox
[thermal fix] add unit test for loading thermal policy file with dupl…
4e689ac 
…icate conditions in different policies
@Junchao-Mellanox
[thermal] fix fan.get_presence for non-removable SKU
0b45c45
@Junchao-Mellanox
[thermal fix] fix issue: fan direction is based on drawer
bdfc652
@Junchao-Mellanox
Fix issue: when fan is not present, should not read fan direction fro…
80e0b88 
…m sysfs but directly return N/A
@Junchao-Mellanox
[thermal fix] add unit test for get_direction for absent FAN
5b46d6f
@Junchao-Mellanox
Unplugable PSU has no FAN, no need add a FAN object for this PSU
2d42cd3
@Junchao-Mellanox
1. Enable thermal alogrithm by default; 2. set cooling level before s…
6f80098 
…et speed; 3. check thermal zone temperature before set speed; 4. enable thermal algorithm by change thermal_zone_mode
@Junchao-Mellanox
start thermal algorithm should also check thermal zone temperature
d891200
@Junchao-Mellanox
Should write string to file
71c3665
@Junchao-Mellanox
We should force enable or disable thermal algo when thermal control d…
f409195 
…aemon start up
@Junchao-Mellanox
Change thermal algorithm status should also change thermal zone policy
12a9d8f
@Junchao-Mellanox
Merge remote-tracking branch 'origin/master' into thermal-algo-fix
527679a 
Conflicts:
	platform/mellanox/mlnx-platform-api/sonic_platform/thermal.py
	platform/mellanox/mlnx-platform-api/tests/test_thermal_policy.py
@Junchao-Mellanox
Add fan speed dynamic minimum value
766aff0
@Junchao-Mellanox
Add unit test for DynamicMinCoolingLevelPolicy
1bcec63
@Junchao-Mellanox
If current cooling state below minimum cooling state, set it to minim…
c6ed366 
…um cooling state
keboliu
keboliu reviewed Mar 29, 2020
View reviewed changes
Copy link
Collaborator

@keboliu keboliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one general comment: I see dynamic minimal is already here, do you want to submit this change with others since we need to align it with Dvir? another thought is that we can separate dynamic minimal and submit other fixes first?

@Junchao-Mellanox
Copy link
Owner Author

Junchao-Mellanox commented Mar 30, 2020 •
edited
Loading

one general comment: I see dynamic minimal is already here, do you want to submit this change with others since we need to align it with Dvir? another thought is that we can separate dynamic minimal and submit other fixes first?

I think we should submit the change without dynamic minimal, if this internal PR is reviewed and approved, my plan is to create a community PR without dynamic minimal. What's your suggestion?

Junchao-Mellanox pushed a commit that referenced this pull request Apr 22, 2020
@taocy001
[devices]: Centec x86 goldengate platforms and devices update (sonic-…
744d33d 
…net#4248)

* Centec x86 goldengate update (#8)
Junchao-Mellanox pushed a commit that referenced this pull request Dec 14, 2020
@dflynn-Nokia
[sonic-mgmt-common] Update submodule (sonic-net#6129)
98a8753 
This update brings in the following commits.

86c1108 Enable arm architecture to build in addition to amd64 (#37)
4acb2c3 fix bugs and enhance Transformer (#35)
49e5a22 ygot related enhancements and fixes (#34)
51224de Fix ietf yang search path for cvl schema builds (#32)
3c6cdb3 CVL Changes #8: 'must' and 'when' expression evaluation (#31)
dabf231 CVL Changes #7: 'leafref' evaluation (#28)
6f9535f CVL Changes #6: Customized Xpath Engine integration (#27)
5e2466b DB-Layer fixes/enhancements (#26)
9a27302 CVL Changes #4: Implementation of new CVL APIs (#22)
dbf1093 Translib support for authorization, yang versioning and Delete flag (#21)
80f369e CVL Changes #5: YParser enhancement (#23)
904ce18 CVL Changes #3: Multi-db instance support (#20)
9d24a34 CVL Changes #2:  YValidator infra changes for evaluating xpath expression (#19)
f3fc40f CVL Changes #1: Initial CVL code reorganization and common infra changes (#18)
4922601 Bulk and RPC API support in translib (#16)
1d730df RFC7895 yang module library implementation (#15)
Junchao-Mellanox pushed a commit that referenced this pull request Nov 12, 2021
@KostiantynYarovyiBf @saiarcot895
Update Barefoot platform module for Bullseye and 5.10 kernel (#8)
9894116 
Update Barefoot platform support for Bullseye and 5.10 kernel, and add
python3-venv.
Junchao-Mellanox pushed a commit that referenced this pull request Dec 7, 2021
@zjswhhh
[submodule]: Update submodule sonic-linkmgrd (sonic-net#9421)
f93d1f6 
6c6151b Fix unstable unit tests (state change handler wasn't invoked) (#8)
2f7dc0a support code diff coverage (#5)
83f0002 Force mux state switch to standby if triggered from Cli (#6)

signed-off-by: Jing Zhang zhangjing@microsoft.com
Junchao-Mellanox pushed a commit that referenced this pull request Dec 9, 2021
@zjswhhh
[sonic-linkmgrd][master] submodule update (sonic-net#9420)
a263edb 
#### What I did 

[sonic-linkmgrd][master] submodule update

6c6151b Fix unstable unit tests (state change handler wasn't invoked) (#8)
2f7dc0a support code diff coverage (#5)
83f0002 Force mux state switch to standby if triggered from Cli (#6)

signed-off-by: Jing Zhang zhangjing@microsoft.com
Junchao-Mellanox pushed a commit that referenced this pull request Jan 17, 2022
@KostiantynYarovyiBf @VadymYashchenko
@dmytroxIntel @vboykox @bratashX @mykolaxgerasymenko
[BFN] Updated platform plugins (sonic-net#9540)
4037867 
* [BFN] Updated platform APIs impl

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* Extended BFN platform SFP APIs implementation

* Update sfp.py

* [BFN] Extended SFP platform plugin implementation

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* [BFN] Extended Fans platform plugin implementation

* [BFN] divided classes Fan and  FanDrawer into 2 files

* Signed-off-by: Vadym Yashchenko <vadymx.yashchenko@intel.com>

What I did
	Add get_model() function
	Add get_low_critical_threshold() function
	Change __get(...) function.
How I did it
	Differnece from previous implementation of __get(...) function is return real value or -9999.9 if value is not provided by thrift API

* Add get_presence() function and revised __get() function

Signed-off-by: Vadym Yashchenko <vadymx.yashchenko@intel.com>

* [BFN] Updated PSU platform APIs impl

Signed-off-by: Dmytro Lytvynenko <dmytrox.lytvynenko@intel.com>

* Added BFN PSU cache (#9)

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* [BFN]  Fans and Fantray platform APIs update (#7)

* [BFN] Updated SFP platform APIs (#10)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>

* [BFN] Updated platform API for thermal (#8)

* Signed-off-by: Vadym Yashchenko <vadymx.yashchenko@intel.com>

* Revert "[BFN]  Fans and Fantray platform APIs update (#7)" (#11)

This reverts commit c62a733.

* Add support health monitor system (#15)

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>

* Update chassis.py

* [BFN] Updated FANs and FAN Tray platform API (#14)

* Fix fix_alignment (#17)

Signed-off-by: Petro Bratash <petrox.bratash@intel.com>

* [BFN] Improvement show environment (#16)

* Added PSU temperature skip into platform.json (#18)

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* Do not skip psud on Newport

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* [BFN] fix fan status from Not OK to Ok (#19)

* [BFN] Updated SFP platform plugin (#13)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>

* [DPB] Fix typo for Ethernet0 2x200G[100G,40G] breakout mode (#21)

Signed-off-by: Mykola Gerasymenko <mykolax.gerasymenko@intel.com>

* [barefoot] Tmp fix vendor_rev (#22)

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>

* Fixed python issues in sonic_platform/fan_drawer.py

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* Updated fan_drawer.py

* Fixing trailing white spaces in fan_drawer.py

* [BFN] Fix thrift for SFPs API

Signed-off-by: Volodymyr Boyko <volodymyrx.boiko@intel.com>

* In platform.json, replaced 'false' with '0' to workaround ast.literal_eval() issue

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* [Newport] Thermal manager  (#23)

* Signed-off-by: Vadym Yashchenko <vadymx.yashchenko@intel.com>

* Revert "In platform.json, replaced 'false' with '0' to workaround ast.literal_eval() issue"

This reverts commit 1e73127.

* Removed 'controllable' options from platform.json to fix factory default config generation

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

* Update thermal_manager.py

* Migrated SFP plugin to sonic_xcvr API (#30)

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

Co-authored-by: KostiantynYarovyiBf <kostiantynx.yarovyi@intel.com>
Co-authored-by: Vadym Yashchenko <vadymx.yashchenko@intel.com>
Co-authored-by: Dmytro Lytvynenko <dmytrox.lytvynenko@intel.com>
Co-authored-by: Volodymyr Boiko <volodymyrx.boiko@intel.com>
Co-authored-by: Petro Bratash <petrox.bratash@intel.com>
Co-authored-by: Mykola Gerasymenko <mykolax.gerasymenko@intel.com>
Junchao-Mellanox pushed a commit that referenced this pull request Dec 11, 2024
@dgsudharsan
[FRR]Fixing zebra to use internal rbtree per NS tree of ifps (sonic-n…
c715dff 
…et#21095)

Adding the below fix from FRR FRRouting/frr#17297

This is to fix the following crash which is a statistical issue

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/lib/frr/zebra -A 127.0.0.1 -s 90000000 -M dplane_fpm_nl -M snmp'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007fccd7351e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7fccd6faf7c0 (LWP 36))]
(gdb) bt
#0  0x00007fccd7351e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fccd7302fb2 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fccd72ed472 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fccd75bb3a9 in _zlog_assert_failed (xref=xref@entry=0x7fccd7652380 <_xref.16>, extra=extra@entry=0x0) at ../lib/zlog.c:678
#4  0x00007fccd759b2fe in route_node_delete (node=<optimized out>) at ../lib/table.c:352
#5  0x00007fccd759b445 in route_unlock_node (node=0x0) at ../lib/table.h:258
#6  route_next (node=<optimized out>) at ../lib/table.c:436
#7  route_next (node=node@entry=0x56029d89e560) at ../lib/table.c:410
#8  0x000056029b6b6b7a in if_lookup_by_name_per_ns (ns=ns@entry=0x56029d873d90, ifname=ifname@entry=0x7fccc0029340 "PortChannel1020")
    at ../zebra/interface.c:312
#9  0x000056029b6b8b36 in zebra_if_dplane_ifp_handling (ctx=0x7fccc0029310) at ../zebra/interface.c:1867
#10 zebra_if_dplane_result (ctx=0x7fccc0029310) at ../zebra/interface.c:2221
#11 0x000056029b7137a9 in rib_process_dplane_results (thread=<optimized out>) at ../zebra/zebra_rib.c:4810
#12 0x00007fccd75a0e0d in thread_call (thread=thread@entry=0x7ffe8e553cc0) at ../lib/thread.c:1990
#13 0x00007fccd7559368 in frr_run (master=0x56029d65a040) at ../lib/libfrr.c:1198
#14 0x000056029b6ac317 in main (argc=9, argv=0x7ffe8e5540d8) at ../zebra/main.c:478
Junchao-Mellanox pushed a commit that referenced this pull request Dec 24, 2024
@dgsudharsan
[202405][FRR] Fixing FRR to make route node lock (sonic-net#20990)
fef2d0d 
To fix a statistical issue. The original fix was done in FRRouting/frr#17297. However to accommodate 8.5.4 the patch in the PR was added.

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/lib/frr/zebra -A 127.0.0.1 -s 90000000 -M dplane_fpm_nl -M snmp'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007fccd7351e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7fccd6faf7c0 (LWP 36))]
(gdb) bt
#0  0x00007fccd7351e2c in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fccd7302fb2 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fccd72ed472 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fccd75bb3a9 in _zlog_assert_failed (xref=xref@entry=0x7fccd7652380 <_xref.16>, extra=extra@entry=0x0) at ../lib/zlog.c:678
#4  0x00007fccd759b2fe in route_node_delete (node=<optimized out>) at ../lib/table.c:352
#5  0x00007fccd759b445 in route_unlock_node (node=0x0) at ../lib/table.h:258
#6  route_next (node=<optimized out>) at ../lib/table.c:436
#7  route_next (node=node@entry=0x56029d89e560) at ../lib/table.c:410
#8  0x000056029b6b6b7a in if_lookup_by_name_per_ns (ns=ns@entry=0x56029d873d90, ifname=ifname@entry=0x7fccc0029340 "PortChannel1020")
    at ../zebra/interface.c:312
#9  0x000056029b6b8b36 in zebra_if_dplane_ifp_handling (ctx=0x7fccc0029310) at ../zebra/interface.c:1867
#10 zebra_if_dplane_result (ctx=0x7fccc0029310) at ../zebra/interface.c:2221
#11 0x000056029b7137a9 in rib_process_dplane_results (thread=<optimized out>) at ../zebra/zebra_rib.c:4810
#12 0x00007fccd75a0e0d in thread_call (thread=thread@entry=0x7ffe8e553cc0) at ../lib/thread.c:1990
#13 0x00007fccd7559368 in frr_run (master=0x56029d65a040) at ../lib/libfrr.c:1198
#14 0x000056029b6ac317 in main (argc=9, argv=0x7ffe8e5540d8) at ../zebra/main.c:478
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@keboliu keboliu keboliu left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Junchao-Mellanox @keboliu @stephenxs