Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

control-plane: Add possibility to manage consul sidecar resources with annotations #956

Merged
merged 1 commit into from
Jan 19, 2022

Conversation

rrondeau
Copy link
Contributor

Hi

We are currently migrating our metrics polling to use merged metrics and we have one issue.
Some of our pods return lots of metrics (~150k) and the consul sidecar get OOM killed.
We bump the memory of the sidecar but dont want to waste memory where we dont need it.

Changes proposed in this PR:

  • default resources are set via flags of the injector
  • can be overriden with annotations on pods

I renamed the injector flags to mirror the envoy sidecar configuration.
The code is heavily inspired by envoy sidecar configuration but i kept the consulSidecarResources object in the subcommand.
If you want that i replicate the envoy stuff and split the resources into 4 vars when passing the settings to the handler, i can do it.

I am open to any ideas / modifications :)

How I've tested this PR:

  • Added tests
  • Run the tests ( control-plane and charts )

How I expect reviewers to test this PR:
¯\(ツ)

Checklist:

  • Tests added
  • CHANGELOG entry added

    HashiCorp engineers only, community PRs should not add a changelog entry.
    Entries should use present tense (e.g. Add support for...)

…h annotations

- default resources are set via flags of the injector
- can be overriden with annotations on pods
@david-yu
Copy link
Contributor

Thanks for the PR @rrondeau this sounds very useful. I'm curious how is it that your pod emits 150k metrics? That is definitely something that we did not consider before!

@rrondeau
Copy link
Contributor Author

Yeah we have 2 types of pod that need more than 50Mi of memory.

  • nodejs app + envoy metrics with ~ 20 upstreams
$ curl -s 127.0.0.1:20200/metrics | grep ^envoy_ | wc -l
11461
  • nodejs app with lots of custom metrics for our business/internal usage
/ $ curl -s 127.0.0.1:20200/metrics | grep -v -e ^envoy_ -e ^# -e ^$ | wc -l
138533

Copy link
Contributor

@ishustava ishustava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@ishustava ishustava requested review from a team, ndhanushkodi and t-eckert and removed request for a team and t-eckert January 14, 2022 20:32
Copy link
Contributor

@ndhanushkodi ndhanushkodi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants