Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add hugepages into the HostMemoryFull alert expression #230

Merged
merged 5 commits into from
Jan 14, 2025

Conversation

lucabello
Copy link
Contributor

@lucabello lucabello commented Jan 13, 2025

Issue

Closes #41.

Solution

Warning

I had to bump the cos_agent library because of a linting error which was auto-fixed by tox -e fmt.

Modify the alert expression as detailed in the issue. The expression was flipped because there is a MemFree_bytes metric, but not a MemUsed_bytes one.

TL;DR (see comments in the yaml below):

expr: |
      100 * avg_over_time(node_memory_MemFree_bytes[1m]) /
        (
          avg_over_time(node_memory_MemTotal_bytes[1m])
          - avg_over_time(node_memory_Hugetlb_bytes[1m])
        ) < 10

Here's a visual representation:

🔴 Pre-allocated HugePages memory (node_memory_Hugetlb_bytes)
🟣 Memory used outside of HugePages (node_memory_MemTotal_bytes - node_memory_Hugetlb_bytes)
🔵 Available memory (node_memory_MemFree_bytes)

🔴 + 🟣 = node_memory_MemTotal_bytes

Memory: 🔴🔴🔴🔴🔴🔴🔴🟣🟣🟣🟣🔵🔵

The alert is checking that 🔵/🟣 is less than 10%.

@lucabello lucabello requested a review from a team as a code owner January 13, 2025 13:37
@lucabello lucabello merged commit 4972c50 into main Jan 14, 2025
12 checks passed
@lucabello lucabello deleted the feature/hugepages-memory-alert branch January 14, 2025 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HostMemoryFull alert doesn't take into account available hugepages
2 participants