Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test_geo* #39

Closed
aiven-sal opened this issue Jul 9, 2024 · 2 comments · Fixed by #61
Closed

Flaky test_geo* #39

aiven-sal opened this issue Jul 9, 2024 · 2 comments · Fixed by #61
Assignees
Labels
flaky-test good first issue Good for newcomers help wanted Extra attention is needed

Comments

@aiven-sal
Copy link
Member

Some tests that match test_geo* in tests/test_commands.py and tests/test_asyncio/test_commands.py are not reliable.
We never observed them failing in CI or locally, but I received reports that they failed in some occasions.

These tests expect exact matches between floats e.g.

def test_geopos(self, r):
values = (2.1909389952632, 41.433791470673, "place1") + (
2.1873744593677,
41.406342043777,
"place2",
)
r.geoadd("barcelona", values)
# valkey uses 52 bits precision, hereby small errors may be introduced.
assert_resp_response(
r,
r.geopos("barcelona", "place1", "place2"),
[
(2.19093829393386841, 41.43379028184083523),
(2.18737632036209106, 41.40634178640635099),
],
[
[2.19093829393386841, 41.43379028184083523],
[2.18737632036209106, 41.40634178640635099],
],
)

And this doesn't always work.

According to the reports I received, the values are usually off by ~1e-15

Floats should be compared using math.isclose. The default tolerance should be enough.

@aiven-sal
Copy link
Member Author

aiven-sal commented Jul 11, 2024

These are the tests that have been reported as failing:

FAILED tests/test_commands.py::TestValkeyCommands::test_geopos - assert [(2.1...
FAILED tests/test_commands.py::TestValkeyCommands::test_geosearch_member - As...
FAILED tests/test_commands.py::TestValkeyCommands::test_geosearch_with - Asse...
FAILED tests/test_commands.py::TestValkeyCommands::test_geosearchstore_dist
FAILED tests/test_commands.py::TestValkeyCommands::test_georadius_with - Asse...
FAILED tests/test_commands.py::TestValkeyCommands::test_georadius_store_dist
FAILED tests/test_commands.py::TestValkeyCommands::test_georadiusmember - Ass...
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_geopos[single]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_geopos[pool]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadius_with[single]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadius_with[pool]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadius_store_dist[single]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadius_store_dist[pool]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadiusmember[single]
FAILED tests/test_asyncio/test_commands.py::TestValkeyCommands::test_georadiusmember[pool]

here is the full build log: https://koji.fedoraproject.org/koji/taskinfo?taskID=120194663

The error happens on s390x, it may be impossible to reproduce on other architectures.

@aiven-sal aiven-sal added help wanted Extra attention is needed good first issue Good for newcomers labels Jul 11, 2024
@kjaymiller
Copy link
Contributor

kjaymiller commented Jul 18, 2024

would testing on a docker image as mentioned in the link below work?

https://docs.gitlab.com/omnibus/development/s390x.html#vm-provisioning

I should have a PR in today or tomorrow and can take a little more time to try and run on this if you believe it will effectively test.

[UPDATE]

I seem to be having the same test failures on my mac (m2 pro). So perhaps we don't need an s390x to verify the tests now pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants