Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shardsManager: fix unchanged shard assignment causes exception #171

Merged
merged 4 commits into from
Jul 17, 2024

Conversation

mattisonchao
Copy link
Member

@mattisonchao mattisonchao commented Jul 16, 2024

Issue

#170

RCA

Oxia-java client didn't handle well with unchanged assignments. which calls the toMap method with the duplicated key and causes an exception throw in the GRPC stream.

The real exception capture by Arthas

java.lang.IllegalStateException: Duplicate key 36 (attempted merging values Shard[id=36, leader=***********, hashRange=HashRange[minInclusive=0, maxInclusive=4294967295]] and Shard[id=36, leader=***********, hashRange=HashRange[minInclusive=0, maxInclusive=4294967295]])

Why can the client receive notification about unchanged assignments?

  1. The coordinator sends node-level assignments to the oxia server.
  2. The oxia server will filter the namespace without an equals check and then send it to the client. (maybe caused by cost/data consistent consideration)

Modification

  • Add merge function to avoid exception
  • Print the root cause messages.

@mattisonchao mattisonchao force-pushed the fix/assignments_update branch from 1ba14a0 to 6420103 Compare July 16, 2024 14:27
@mattisonchao mattisonchao linked an issue Jul 16, 2024 that may be closed by this pull request
@merlimat merlimat merged commit 3566189 into main Jul 17, 2024
1 check passed
@merlimat merlimat deleted the fix/assignments_update branch July 17, 2024 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

constantly throw Failed receiving shard assignments exception
3 participants