Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle arbitrary types in bloom filter synopsis #1685

Merged
merged 2 commits into from
May 27, 2021

Conversation

lava
Copy link
Member

@lava lava commented May 26, 2021

📔 Description

The meta index lookup would select type synopses based on the field name being looked up, without verifying that the right hand side of the query also has the corresponding type. This leads to undefined behavior for queries like s == nil or s == 123 (where s is a string field) and depending on the compiler settings also segfaults or other kinds of bad behavior.

NOTE: I'm not completely sure if the same issue exists for "normal" synopsis and we just don't notice it because the invalid call to caf::get<view<T>>() was not using complex types like std::string, or if these were typechecked somewhere else beforehand.

📝 Checklist

  • All user-facing changes have changelog entries.
  • The changes are reflected on docs.tenzir.com/vast, if necessary.
  • The PR description contains instructions for the reviewer, if necessary.

🎯 Review Instructions

@dominiklohmann dominiklohmann added the bug Incorrect behavior label May 26, 2021
@lava lava force-pushed the story/ch25631/nil-segfault branch from 84b48ce to 62c404a Compare May 26, 2021 14:35
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this looks fine to me, but I would like to refrain from merging this change with such short notice before the release. This does not fix a bug that was just recently introduced, but rather a long-standing issue and we must properly evaluate whether this is the proper fix, or whether we want to look at using transparent hashing for the bloom filter.

@dominiklohmann dominiklohmann added the blocked Blocked by an (external) issue label May 26, 2021
@dominiklohmann dominiklohmann removed the blocked Blocked by an (external) issue label May 27, 2021
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with @tobim. Needs a rebase + a story for switching the result type of the lookup from optional<bool> to an enum class with three members.

@lava lava force-pushed the story/ch25631/nil-segfault branch from 3950b32 to 4990718 Compare May 27, 2021 15:36
@lava
Copy link
Member Author

lava commented May 27, 2021

I've rebased the PR. The story (in a slightly more general form) exists as https://app.clubhouse.io/tenzir/story/19453/discussion-change-meta-index-api-to-enable-short-circuiting

@lava lava merged commit 2d85ddb into master May 27, 2021
@lava lava deleted the story/ch25631/nil-segfault branch May 27, 2021 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants