Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event queue processes and filling race. #253

Merged
merged 1 commit into from
Jul 25, 2019

Conversation

tcolgate
Copy link
Contributor

We send the slice off to be process, but potentially start overwriting
the previous content straight away. Just start again with a fresh slice.

Fixes #247

@tcolgate
Copy link
Contributor Author

It make be that it would be better to create a new sized slice to avoid the subsequent allocations during growth.

@claytono
Copy link
Contributor

@tcolgate Agreed. The expectation in the existing code is that the slice is going to grow to the size needed, and once it does, future memory allocations can be avoided. Making a copy before sending it over the channel seems like the way to go.

@matthiasr
Copy link
Contributor

That makes sense, a single reasonably-sized allocation is better than many smaller ones ending in the same size. We're trading a tiny bit of memory usage for low-traffic use cases but I think that is acceptable.

@tcolgate
Copy link
Contributor Author

tcolgate commented Jul 25, 2019

I see two options:

  • make a new slice with the cap of the old one
  • Always initialize the slice cap to flushThreshold and be done with it.

I vote for option 2 (for a busy server, I guess the degenerate to the same thing)

@matthiasr
Copy link
Contributor

Always initialize the slice cap to flushThreshold and be done with it.

I think that's the best way.

We send the slice off to be process, but potentially start overwriting
the previous content straight away. Just start again with a fresh slice.

Fixes prometheus#247

Signed-off-by: Tristan Colgate <tristan@qubit.com>
@tcolgate
Copy link
Contributor Author

@matthiasr I changed my mind, or rather, maybe we do both. This allocates the initial slice, and then just keeps copying the cap after that. This will cause something like 160KB of extra allocation on startup by default, I don't think people will notice.

@matthiasr matthiasr merged commit 353b38c into prometheus:master Jul 25, 2019
@matthiasr
Copy link
Contributor

Thank you!

matthiasr pushed a commit that referenced this pull request Jul 25, 2019
Signed-off-by: Matthias Rampke <mr@soundcloud.com>
@tcolgate tcolgate deleted the eventrace branch July 26, 2019 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

panic: runtime error: makeslice: cap out of range
3 participants