Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamline case sensitive behavior of PropertyValueCriteria #4721

Closed
2 tasks done
bwaidelich opened this issue Nov 10, 2023 · 3 comments · Fixed by #4725
Closed
2 tasks done

Streamline case sensitive behavior of PropertyValueCriteria #4721

bwaidelich opened this issue Nov 10, 2023 · 3 comments · Fixed by #4725
Labels

Comments

@bwaidelich
Copy link
Member

bwaidelich commented Nov 10, 2023

Currently the PropertyValueCriteriaInterface implementations make no assumption on whether the properties are compared in case sensitive or insensitive manner.

Tasks

  • Document current behavior and add tests
  • Consider adding support for explicit case sensitive mode (see related PR)
@bwaidelich bwaidelich added the 9.0 label Nov 10, 2023
@bwaidelich
Copy link
Member Author

bwaidelich commented Nov 10, 2023

Copied from Slack:

regarding properties and case sensitivity I see a lot of use of JSON_SEARCH in the codebase, for which documentation is spotty at best but this suggests it's case sensitive by default and would need a lowercase wrapping to make it case insensitive https://stackoverflow.com/questions/63242596/how-to-make-the-json-search-not-case-sensitive

Usually case (in)sensitivity depends on the collation (e.g. the "ci" in utf8_unicode_ci stands for case insensitive).

I tested the following with MariaDB 10.8:

SELECT
  JSON_EXTRACT(properties, '$.title.value')
FROM
  cr_default_p_graph_node
WHERE
  JSON_SEARCH(properties, 'one', 'home', NULL, '$.title.value') IS NOT NULL

Will return nodes with a title property of "Home" (upper case).
But if I change the WHERE part to

JSON_SEARCH(properties COLLATE utf8mb3_bin, 'one', 'home', NULL, '$.title.value') IS NOT NULL

Only nodes matching the exact case are returned.

For MySQL (8.0.33) the default behavior seems to be different and JSON_SEARCH works case sensitive
So to make that case insensitive we'd have to cast the incoming string like:

JSON_SEARCH(properties, 'one', 'home' COLLATE utf8mb4_unicode_ci, NULL, '$.title.value') IS NOT NULL;

=> I would suggest that we always specify the collation (for now _bin on both to reproduce todays case sensitive behavior, later depending on the flag if that is added to the PropertyValueCriteria)

Note: I'm a bit puzzled that the cr_default_p_graph_node is based on utf8mb3 and the incoming parameters on utf8mb4 (that's why I used utf8mb3_bin and utf8mb4_unicode_ci above for the casting)

For PostgreSQL everything is slightly different, but I imagine the solution to be similar.

that leads me to another question, JSON_SEARCH is (my|maria) specific so the dbal adapter is not compatible with anything but the two?

Yes, for now the adapter supports MariaDB and MySQL (and potentially SQLite?)

We'll have to double check that (and also the supported versions), document it and fail early if the platform is not supported (see #4337)

@kitsunet
Copy link
Member

Great, thanks for testing this out! Sooo weird that they behave differently!

bwaidelich added a commit that referenced this issue Nov 10, 2023
Streamlines the behavior of the `ContentSubgraph` for string based
PropertyValueCriteria to be case sensitive (to be in sync with Neos < 8)

Related: #4721
@bwaidelich bwaidelich moved this to Under Review 👀 in Neos 9.0 Release Board Nov 10, 2023
@mficzel
Copy link
Member

mficzel commented Nov 11, 2023

Nut sure wether i like a flag or distinct criteria classes better since =~ is clearly different from =. In the end it does not matter that much so whatever gets build would be fine with me.

@github-project-automation github-project-automation bot moved this from Under Review 👀 to Done ✅ in Neos 9.0 Release Board Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants