Add support for PFSELFTEST command

[Vsareen0]

Add support for the PFSELFTEST command in DiceDB similar to the PFSELFTEST command in Redis. Please refer to the following commit in Redis to understand the implementation specifics - source.

Write unit and integration tests for the command referring to the tests written in the Redis codebase 7.2.5. For integration tests, you can refer to the tests folder. Note: they have used TCL for the test suite, and we need to port that to our way of writing integration tests using the relevant helper methods. Please refer to our tests directory.

For the command, benchmark the code and measure the time taken and memory allocs using benchmem and try to keep them to the bare minimum.

[AshwinKul28]

@Vsareen0 Any updates on this?

[AshwinKul28]

Hey @Vsareen0 Sorry to bother you again, Do you have any updates on this? If you have any blockers lets discuss this over the discord. Or if you have multiple items on the plate, lets open this issue for everyone, and others can pick this up.
Thanks

cc - @JyotinderSingh

[chettriyuvraj]

Can pick this up if possible!

[AshwinKul28]

@chettriyuvraj Thanks, go for it.

[chettriyuvraj]

Sorry for no updates on this one - was down with H1N1 the past week. Will pick up and put out an update @AshwinKul28.

[ankkyprasad]

Hey @chettriyuvraj, I hope you're doing well. Are you still working on this issue? If not, I can take this up.

[chettriyuvraj]

Hey @ankkyprasad! I am - will put an update up shortly

[chettriyuvraj]

Picking this issue in conjunction with #446 so the initial update is literally the same.

Today was the first day I actively picked up the issue. I'll be posting an update daily from now on.

Status

I mentioned that I had no clue about what HyperLogLog was and my first step today was to pick up the paper.

• Went over the paper
• The idea of cardinality estimation structures was new to me but seemed very intuitive.
• The proofs/math zoomed over my head.
• After getting the overview wanted some more concreteness to my understanding - quickly ran over this video, this article.
• The top level comment in Redis's implementation for understanding sparse and dense representations

I'll hopefully have a bit more concrete updates + queries to ask tomorrow.

[chettriyuvraj]

Status

Quickly mapped out things on paper to figure if my understanding of the baseline concepts were clear:

• Registers: mapping out bits on paper and seeing if my understanding of how cardinalities are estimated is correct
• Sparse/dense representation

I think thats all I need to figure this issue - will look at the implementation of PFSELFTEST in Redis source tomorrow and hopefully be able to make sense of it

[chettriyuvraj]

Status

Hi @AshwinKul28!

Went over the source for pfselftest command

PFSELFTEST performs a test of redis's HLL register implementation We are using a library for our HLL needs.

1. The internals of the library such as the register values aren't exposed, how do we implement pfselftest in this case?
2. Will implementing pfselftest make sense since we would be testing the library implementation?

[arpitbbhayani]

Hello @chettriyuvraj,

There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

[arpitbbhayani]

Hello @chettriyuvraj,

There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

[arpitbbhayani]

Hello @chettriyuvraj,

There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.

We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.

Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.

Thanks again.

[chettriyuvraj]

Hi @arpitbbhayani - have unassigned myself from this issue. Free to assign it to someone else.

[helios2003]

Hi @arpitbbhayani, @lucifercr07 can I work on this issue?

[lucifercr07]

@helios2003 assigned, thanks for contributing.

[helios2003]

Hi, @lucifercr07! I've been busy over the past few days, but I took some time to look into how to approach the issue. I'll reach out here or on Slack with any questions or doubts that come up in the next few days. Thanks for your support!