Skip to content
This repository has been archived by the owner on Apr 4, 2023. It is now read-only.

Prefer returning None instead of using an FilterCondition::Empty state #422

Merged
merged 5 commits into from
Dec 9, 2021

Conversation

Kerollmops
Copy link
Member

@Kerollmops Kerollmops commented Dec 9, 2021

This PR is related to the issue comment meilisearch/meilisearch#1338 (comment) which exhibits the fact that when a filter is known to be empty no results are returned which is wrong, the filter should not apply as no restriction is done on the documents set.

The filter system on the milli side has introduced an Empty state which was used in this kind of situation but I found out that it is not needed and that when we parse a filter and that it is empty we can simply return None as the Filter::from_array constructor does. So I removed it and added tests!

On the MeiliSearch side, we just need to match on a None and completely ignore the filter in such a case.

@bidoubiwa
Copy link
Contributor

I answered on the related issue with what I think raises my point. meilisearch/meilisearch#1338 (comment)
With this PR it fixes the problem I raised?

@Kerollmops Kerollmops requested a review from irevoire December 9, 2021 12:09
@Kerollmops Kerollmops added the DB breaking The related changes break the DB label Dec 9, 2021
@curquiza
Copy link
Member

curquiza commented Dec 9, 2021

@meilisearch/product-team can you confirm this is the behavior you expect?

@Kerollmops
Copy link
Member Author

Kerollmops commented Dec 9, 2021

This is an issue that seems to have been raised when we switch from pest to nom, we used the Filter::Empty variant instead of returning an optional which breaks this behavior. It should never have behaved like this and don't restrict the results at all instead of returning nothing when the filter is empty.

The Filter::Empty variant was used in the previous parser version when a field name was unknown by the engine which means that no documents were matching this filter condition, as the field doesn't exist it matches no documents. This is no more required as the filter parser doesn't understand the unknown fields until it is given to the milli engine which returns an error when it happens.

@curquiza curquiza requested a review from gmourier December 9, 2021 14:21
@Kerollmops
Copy link
Member Author

Thank you for the review!
bors merge

@bors
Copy link
Contributor

bors bot commented Dec 9, 2021

@bors bors bot merged commit 11a056d into main Dec 9, 2021
@bors bors bot deleted the empty-filters branch December 9, 2021 15:12
@gmourier
Copy link
Member

gmourier commented Dec 9, 2021

@meilisearch/product-team can you confirm this is the behavior you expect?

This is consistent with the behavior of other fields like attributesToRetrieve which can be sent as [] for example. Since we have several possible filter syntaxes, it makes sense to be able to specify "" as [] for this field.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
DB breaking The related changes break the DB
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants