Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: precise query id rollover semantic #1606

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

wyfo
Copy link
Contributor

@wyfo wyfo commented Nov 22, 2024

Allow managing rollover and uniqueness, as well as being faster to access.

@wyfo wyfo requested review from Mallets and OlivierHecart and removed request for Mallets November 22, 2024 06:59
Copy link

PR missing one of the required labels: {'enhancement', 'dependencies', 'breaking-change', 'internal', 'documentation', 'bug', 'new feature'}

@wyfo wyfo added the enhancement Existing things could work better label Nov 22, 2024
wyfo added 2 commits November 22, 2024 13:31
Allow managing rollover and uniqueness, as well as being faster to access.
@wyfo wyfo force-pushed the fix/rollover_uniqueness branch from 08e13ba to a2b1583 Compare November 22, 2024 13:00
@Mallets
Copy link
Member

Mallets commented Nov 28, 2024

@wyfo what's the status of this PR?

@wyfo
Copy link
Contributor Author

wyfo commented Jan 16, 2025

I've realized that just making the query id available right after a timeout is not appropriate, because some response may come after. A better solution in case of timeout would be to add another delay, for example 1min in which the query id cannot be reused. And to handle the case where a lot of queries time out, those pending ids could be be inserted in a dedicated set, to be reused only in case of available id shortage.
@Mallets What do you think about it?

@Mallets
Copy link
Member

Mallets commented Jan 16, 2025

I think the proposed change introduces quite a bit of complexity. We could simply rollover on QueryId assuming the query is no longer alive, which is a realistic assumption given the number of queries required to be performed in parallel.

@wyfo
Copy link
Contributor Author

wyfo commented Jan 16, 2025

I think it's a valid assumption, but it quite defeats the point of having varint encoding for request id.

@Mallets
Copy link
Member

Mallets commented Jan 16, 2025

There might be some room for optimization indeed but I don't believe it's critical.

@wyfo wyfo changed the title fix: use slab instead of hashmap + atomic counter fix: precise query id rollover semantic Jan 17, 2025
@wyfo
Copy link
Contributor Author

wyfo commented Jan 17, 2025

I've updated the PR taking in account the recent discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Existing things could work better
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants