Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--log-usage support in exe mode #227

Open
Jongy opened this issue Nov 27, 2021 · 1 comment
Open

--log-usage support in exe mode #227

Jongy opened this issue Nov 27, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@Jongy
Copy link
Contributor

Jongy commented Nov 27, 2021

--log-usage lets us track the CPU & memory usage of gProfiler. It does so by reading cgroups files for the cgroup gProfiler runs in. See https://github.com/Granulate/gprofiler/blob/89fcf8e271706029bedd486b2b35727e92058086/gprofiler/usage_loggers.py for how it's currently done.

This works in containers, as we can just use the cgroup files of the container.
In "executable" mode, however, we don't run in separate cgroups, so measuring the existing cgroups will not yield the correct result.

If we move ourselves into a child cgroup of the current one, we can then track that new one.

Note: we sometimes run in semi-containers, such as Databricks / Dataflow / Fargate, where we're an executable but we're on low privileges so I don't know if we can create new cgroups. We need to handle this case as well (that is, do not fail hard if we fail to move cgroups)

Resources:

Cgroups are per-thread actually. So we should move our thread to a new cgroup as early as possible (before other threads/processes are spawned), and make sure it's really the only thread in the process once we do so.

@Jongy Jongy added the enhancement New feature or request label Nov 27, 2021
@Jongy
Copy link
Contributor Author

Jongy commented Sep 1, 2022

intel/granulate-utils#56 lets us create & move between cgroups

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant