Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROF-9088] Ensure DSO objects are cleared #400

Merged
merged 7 commits into from
Apr 9, 2024
Merged

Conversation

r1viollet
Copy link
Collaborator

@r1viollet r1viollet commented Apr 4, 2024

What does this PR do?

  • Ensures DSO objects are cleared
  • Move lifetime of DWFL objects into the process object
  • Introduce an input option for maximum PIDs

Motivation

We were seeing a large heap usage with the DSO objects.
Indeed prior to this PR we were not considering the existing DSOs that had no unwinding (dwfl) objects.

Additional Notes

NA

How to test the change?

We should test the RSS improvements on full host applications.

@r1viollet r1viollet force-pushed the r1viollet/fix_leak_dso branch from 056210a to d3128f0 Compare April 4, 2024 07:27
void add_error_frame(const Dso *dso, UnwindState *us,
[[maybe_unused]] ProcessAddress_t pc,
SymbolErrors error_case) {
ddprof_stats_add(STATS_UNWIND_ERRORS, 1, nullptr);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the unwinding error stats was computed on symbolization failures sometimes.

@pr-commenter
Copy link

pr-commenter bot commented Apr 4, 2024

Benchmark results for collatz

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.17.0+7b00f53f.31405626 ddprof 0.17.0+9a07ff97.31694043

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-collatz --preset cpu_only collatz_runner.sh same

@pr-commenter
Copy link

pr-commenter bot commented Apr 4, 2024

Benchmark results for BadBoggleSolver_run

Parameters

Baseline Candidate
config baseline candidate
profiler-version ddprof 0.17.0+7b00f53f.31405626 ddprof 0.17.0+9a07ff97.31694043

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean execution_time
scenario:ddprof -S bench-bad-boggle-solver BadBoggleSolver_run work 1000 same

@r1viollet r1viollet force-pushed the r1viollet/fix_leak_dso branch from 3cc7706 to 9707a09 Compare April 4, 2024 08:52
@r1viollet r1viollet changed the title [ongoing] refactor process objects [PROF-9088] Ensure DSO objects are cleared Apr 7, 2024
@r1viollet r1viollet marked this pull request as ready for review April 7, 2024 08:48
src/ddprof_worker.cc Outdated Show resolved Hide resolved
@r1viollet r1viollet force-pushed the r1viollet/fix_leak_dso branch from 0725404 to 2aefb65 Compare April 7, 2024 18:44
@r1viollet r1viollet force-pushed the r1viollet/fix_leak_dso branch from 2aefb65 to 5c00dcc Compare April 7, 2024 19:06
include/ddprof_defs.hpp Outdated Show resolved Hide resolved
src/unwind.cc Outdated
@@ -59,32 +59,46 @@ void unwind_init_sample(UnwindState *us, const uint64_t *sample_regs,

DDRes unwindstate_unwind(UnwindState *us) {
DDRes res = ddres_init();
Process &process = us->process_hdr.get(us->pid);
bool avoid_new_attach = false;
if (us->process_hdr.process_count() > us->max_pids) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

process_count() might not be equal to the number of attached dwfl objects.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Agreed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not pushing this further as making the control perfect will take quite an extra bit of work. I understand the current limitations.

private:
static std::string format_cgroup_file(pid_t pid,
std::string_view path_to_proc);

static DDRes read_cgroup_ns(pid_t pid, std::string_view path_to_proc,
CGroupId_t &cgroup);

std::unique_ptr<DwflWrapper> _dwfl_wrapper{};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::unique_ptr<DwflWrapper> _dwfl_wrapper{};
std::unique_ptr<DwflWrapper> _dwfl_wrapper;

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not have an opinion on this. Leaving as is unless we push for a different guideline. Feel free to challenge me further if this is important.

include/dwfl_wrapper.hpp Outdated Show resolved Hide resolved
src/ddprof_process.cc Outdated Show resolved Hide resolved
src/ddprof_process.cc Outdated Show resolved Hide resolved
nsavoire
nsavoire previously approved these changes Apr 8, 2024
@r1viollet r1viollet requested a review from nsavoire April 8, 2024 16:07
@r1viollet r1viollet force-pushed the r1viollet/fix_leak_dso branch from b5b82ef to 9a07ff9 Compare April 9, 2024 07:09
Copy link
Collaborator

@nsavoire nsavoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this leak !

@r1viollet r1viollet merged commit 6608480 into main Apr 9, 2024
2 checks passed
@r1viollet r1viollet deleted the r1viollet/fix_leak_dso branch April 9, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants