Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics Collect improvement for delta temporality #2328

Open
cijothomas opened this issue Nov 23, 2024 · 1 comment
Open

Metrics Collect improvement for delta temporality #2328

cijothomas opened this issue Nov 23, 2024 · 1 comment

Comments

@cijothomas
Copy link
Member

During collect the hashmap is cleared out entirely, forcing updates() to re-insert entries to the hashmap after each collect. Since inserts require Write lock and heap allocation, we should avoid unnecessarily causing insertion.

One rough idea how to achieve this:

  1. Associate a status with each tracker indicating if it has a collect pending or not. Update need to set this.
  2. During collect, remove those trackers which has no collect pending.
  3. For trackers that have collect pending, it must be collected and status updated to no collect pending.

This does introduce some extra cost, but it'll help avoid insertion frequency for those trackers that are constantly updated (at least once in collect period), as their trackers will not be removed from hashmap. This is a desirable feature for Metrics, where once a tracker is reported (i.e a timeseries), as long as user is not providing new timeseries, there should be zero memory allocations and write locks. In other words, after initial hydration, the steady state should not require any memory allocations, and won't cause an update thread to get blocked trying to acquire a lock.

@fraillt
Copy link
Contributor

fraillt commented Dec 4, 2024

This has undesirable cost for cumulative temporality as well, because of the requirement to keep track of whether collect is pending or not.
However, it looks like temporality never changes for specific aggregation, which means that instead of having Sum aggregation with collection methods cumulative and delta, we could have Sum<generic over temporality> instead.
This approach would allow to have efficient update for cumulative, and efficient collect for delta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants