Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle field names with any characters with a known set of special #1109

Merged
merged 2 commits into from
Jul 5, 2021

Conversation

fmassot
Copy link
Contributor

@fmassot fmassot commented Jul 3, 2021

…characters and an escape one for the query parser.

Special characters : + ^ ` : { } " [ ] ( ) ~ ! \ * \s.
Almost like Lucene but without && || -.
The escape character is \.
I kept the restriction on the - which cannot be a first character of a field name.
- can be considered as a special character as it is used by the language query, I did not put it in the special characters for retro compatibility.

  • update the query parser to parse such field names
  • update is_valid_field_name function to check only if field name has at least one character and does not start with -

@codecov
Copy link

codecov bot commented Jul 3, 2021

Codecov Report

Merging #1109 (a2d60db) into main (92f20bc) will increase coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head a2d60db differs from pull request most recent head 04c37e5. Consider uploading reports for the commit 04c37e5 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1109      +/-   ##
==========================================
+ Coverage   89.57%   89.59%   +0.02%     
==========================================
  Files         202      202              
  Lines       20198    20210      +12     
==========================================
+ Hits        18092    18108      +16     
+ Misses       2106     2102       -4     
Impacted Files Coverage Δ
query-grammar/src/query_grammar.rs 99.66% <100.00%> (+0.02%) ⬆️
query-grammar/src/user_input_ast.rs 87.67% <100.00%> (ø)
src/schema/mod.rs 100.00% <100.00%> (ø)
src/indexer/merger.rs 89.45% <0.00%> (-0.10%) ⬇️
src/directory/directory.rs 90.24% <0.00%> (+12.19%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92f20bc...04c37e5. Read the comment docs.

@PSeitz
Copy link
Contributor

PSeitz commented Jul 4, 2021

Ah nice the coverage report I added yesterday. Seems to be helpful, but not completely accurate.

@fmassot
Copy link
Contributor Author

fmassot commented Jul 4, 2021

Ah nice the coverage report I added yesterday. Seems to be helpful, but not completely accurate.

very nice indeed, would be nice to add it to quickwit.


/// Parses a field_name
/// A field name must have at least one character and be followed by a colon.
/// All characters are allowed except special characters `SPECIAL_CHARS`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

including special characters, but these needs to be escaped with escape character \.

Copy link
Collaborator

@fulmicoton fulmicoton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I edited a comment and simplified the code a tiny bit)

@fulmicoton fulmicoton merged commit f4b2e71 into main Jul 5, 2021
@fulmicoton fulmicoton deleted the issue/1087 branch July 5, 2021 13:31
This was referenced Feb 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants