Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command migration for single shard, multi-shard and custom commands #1276

Merged
merged 53 commits into from
Nov 13, 2024

Conversation

AshwinKul28
Copy link
Contributor

@AshwinKul28 AshwinKul28 commented Nov 12, 2024

This PR migrates multiple DiceDB commands to support both multi-shard and single-shard scenarios. The migration follows a structured flow for each command type as described below.

Multi-Shard Commands
The following commands are migrated with support for multi-shard processing:

  1. MSET
  2. MGET
  3. SINTER
  4. SDIFF
  5. JSON.MGET

For the COPY and RENAME commands, the flow includes preprocessing, followed by decomposition and composition steps to manage multi-shard data efficiently. For the remaining multi-shard commands, only decomposition and composition steps are involved.

Single-Shard Commands
The following commands are migrated with support for single-shard processing:

  1. JSON.SET
  2. JSON.GET
  3. JSON.TYPE
  4. JSON.INGEST
  5. JSON.HGETALL
  6. JSON.STRAPPEND
  7. Dump
  8. Restore

These single-shard commands follow a scatter-gather approach with a straightforward flow, where each command operates directly within the target shard, without the need for complex decomposition.

@JyotinderSingh
Copy link
Collaborator

Looks like a conflict came up. @AshwinKul28 please feel free to resolve conflict and merge immediately.

Copy link
Contributor

@apoorvyadav1111 apoorvyadav1111 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work, Cant wait to see it merged. left comments more like notes for future.

@AshwinKul28
Copy link
Contributor Author

AshwinKul28 commented Nov 13, 2024

@apoorvyadav1111 @JyotinderSingh any idea why LPush tests are failing? Also HTTP context cancelled error is coming out of nowhere :(

@JyotinderSingh
Copy link
Collaborator

Thanks for the massive effort @AshwinKul28, and thanks for all the debugging help @apoorvyadav1111. We are now much closer to a fully multi-threaded system!

@JyotinderSingh JyotinderSingh merged commit d26d77a into DiceDB:master Nov 13, 2024
2 checks passed
@AshwinKul28 AshwinKul28 deleted the AshwinKul28-master branch November 18, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants