Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#143: Support: Byte Array DataType and Operations #180

Merged
merged 16 commits into from
Aug 4, 2024

Conversation

MohitVachhani
Copy link
Contributor

@MohitVachhani MohitVachhani commented Jul 24, 2024

This PR addresses [Issue]

  • New data type: ByteArray
    • Introduced a length property inside ByteArray to be used whenever we call len().
    • Clearing the data of ByteArray if we decrease its size! [Example]
  • Commands like SETBIT/GETBIT/BITCOUNT/BitOp
    • With unit Test cases.
    • With benchmark test cases.

Note:

  • I have not addressed the BITOP operation in this PR because it is already quite large.

core/eval.go Outdated Show resolved Hide resolved
core/eval.go Outdated Show resolved Hide resolved
@arpitbbhayani
Copy link
Contributor

@yashs360, need your help here. I am in a dilemma.

Given Redis is in C, strings are essentially bytes. In Go, strings are strings and any byte level operation becomes costly.
I was thinking that we diverge from Redis here and offer bit operations on only ByteArray type that you introduced and raise error in case someone is doing it on other data type.

What do you think?

FYI @JyotinderSingh

@MohitVachhani
Copy link
Contributor Author

Hey @arpitbbhayani / @JyotinderSingh / @yashs360

I have added support for:

  • New data type: ByteArray
  • Commands like SETBIT/GETBIT
  • ByteArray usage inside evalBITCOUNT

Doubts:

  • Should we support the string data type for SETBIT/GETBIT commands?
  • ByteArray:
    • Should I introduce a length property inside ByteArray to be used whenever we call len()?
      • This property would be maintained whenever we increase the array size.
    • Do we need to clear the data of ByteArray if we decrease its size?
      • For example, when reducing the size of the array, should we explicitly clear the truncated data?
SETBIT k 7 1
SETBIT K 23 1 
SETBIT K 23 0

So while doing SETBIT K 23 0, should we free up the memory, they are doing in Redis -> Code Reference: This

@yashs360
Copy link
Contributor

yashs360 commented Jul 28, 2024

@arpitbbhayani @MohitVachhani

offer bit operations on only ByteArray

I think its fine as long as we document it. We should be able to diverge from redis as long as we have the capability covered by a the byte commands, and fail for divergence scenarios.

@MohitVachhani
Copy link
Contributor Author

Okay @yashs360 , I will try to add in the comments itself and also will throw different error if any bit operations are done on strings 👍🏾

core/bytearray.go Outdated Show resolved Hide resolved
@@ -31,6 +45,14 @@ func (b *ByteArray) GetBit(pos int) bool {
return (b.data[byteIndex] & (1 << bitIndex)) != 0
}

// GetBit gets the bit at the given position
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question GetBit vs GetBitByByteArrayPosition

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core/eval.go Show resolved Hide resolved
core/eval.go Show resolved Hide resolved
@yashs360
Copy link
Contributor

Nice change @MohitVachhani . Dropped a few comments.

@MohitVachhani
Copy link
Contributor Author

MohitVachhani commented Aug 3, 2024

Hey @yashs360 / @JyotinderSingh,

Could you please let me know the next steps here ? I’m looking to complete this task.

Thanks 🙇🏾

@MohitVachhani MohitVachhani changed the title Feat/eval bitcount Support: Byte Array DataType Aug 3, 2024
@MohitVachhani MohitVachhani changed the title Support: Byte Array DataType Support: Byte Array DataType and Operations Aug 3, 2024
@yashs360
Copy link
Contributor

yashs360 commented Aug 3, 2024

@MohitVachhani Overall the he change looks good, but the rewrite of Set/Get Bits doesn't make sense to me. If there are concerns with existing functions we should modify the existing code to address that instead of creating new functions.
Could you please look at that.

@MohitVachhani
Copy link
Contributor Author

Sure will update the existing function and let you know soon @yashs360

@MohitVachhani
Copy link
Contributor Author

Hey @yashs360 I have updated the code, as per the above comments. Please can you check it out.
Thanks 🙇🏾

Copy link
Collaborator

@JyotinderSingh JyotinderSingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution!

@JyotinderSingh
Copy link
Collaborator

Thanks for the review @yashs360

core/object.go Outdated Show resolved Hide resolved
@JyotinderSingh
Copy link
Collaborator

@MohitVachhani please fix the linter issues. Also added a minor comment.

@JyotinderSingh JyotinderSingh changed the title Support: Byte Array DataType and Operations #143: Support: Byte Array DataType and Operations Aug 4, 2024
@JyotinderSingh JyotinderSingh merged commit 25b3513 into DiceDB:master Aug 4, 2024
2 checks passed
@JyotinderSingh
Copy link
Collaborator

Thanks for addressing the comments and working on this large PR, merged!

SyedMa3 pushed a commit to SyedMa3/dice that referenced this pull request Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants