Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mempool Streaming API #358

Merged
merged 5 commits into from
Jul 29, 2021
Merged

Mempool Streaming API #358

merged 5 commits into from
Jul 29, 2021

Conversation

adityapk00
Copy link
Contributor

New Mempool streaming API and implementation

@adityapk00
Copy link
Contributor Author

cc: @LarryRuane @pacu @gmale

@pacu pacu requested a review from LarryRuane July 15, 2021 19:02
@LarryRuane
Copy link
Collaborator

Hi Aditya, this is great, thanks for contributing this. I tested it and it seems to work perfectly.

Just after you mentioned this idea a month or two ago, I started implementing this myself (sorry for the lack of communication but I wasn't sure how far I'd get), but then I got distracted doing other things and put it aside. Seeing your PR, I looked again at those changes, and decided the best would be to combine our changes, which I did, please take a look at the latest commit on this branch. (I thought it would be simpler to make a commit instead of leaving lots of comments on this PR.) The main difference is that I avoided using goroutines, which are cool but somewhat complex to use and to verify the correctness of (in particular, that there are no race conditions). So the code in my branch is somewhat simpler but very similar. I tested this code but would encourage others to test too.

A small change I made which we may not want is I added a new message type, MempoolTransaction. This is the same as the existing RawTransaction except it doesn't include a height. The comment for the RawTransaction height says that it should be -1 if the transaction is not yet mined. If it's always -1 for this gRPC, we may as well leave it off. I did leave the return as a JSON object (dictionary) so that we can add fields later if needed in a backward-compatible way.

It would also be nice to have unit tests; I can work on those if you'd like.

If @gmale and @pacu haven't started using the inferior (non-streaming) mempool stuff that I pushed a while ago (the GetMempoolTx gRPC), we should also add a commit here to remove that stuff. I doubt anyone else has started using it.

@adityapk00
Copy link
Contributor Author

This is great! I looked through your branch and tested it against Zecwallet, and it looks good to me. We can just use that, and close this PR.

The one request I have is to use RawTransaction (with the height) instead of the MempoolTransaction type. It turns out I need the height (which will not be -1, but the height at which the tx was seen in the mempool), for various reasons in the client.

If you're OK changing the API to return RawTransaction with height = latestHeight, that'll be fantastic!

I'm also OK removing the old mempool API.

@pacu
Copy link
Contributor

pacu commented Jul 19, 2021

@LarryRuane None of the other wallets we support or use our SDK use the mempool RPC

@LarryRuane
Copy link
Collaborator

The one request I have is to use RawTransaction (with the height) ...

Yes, I just restored that (forced-pushed 01812b2).

Instead of closing this PR, it would be less friction to just cherry-pick my commit into this PR (I haven't made my own PR yet, it's just a branch at the moment). Is that okay?

@adityapk00
Copy link
Contributor Author

Sounds good. Will you take over this PR and add the tests as well? I'm not very familiar with Go testing.

Copy link
Collaborator

@LarryRuane LarryRuane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adityapk00, I removed the old mempool code and added a unit test, could you please review? When it looks okay to you, I'll merge it (I don't have any further changes).

@adityapk00
Copy link
Contributor Author

This looks great! Thanks for getting this done!

Looks good to me!

adityapk00 and others added 5 commits July 29, 2021 13:45
This commit is based on adityapk00 streaming mempool interface but
avoids using goroutines, which are difficult to reason about.

Co-authored-by: Aditya Kulkarni <adityapk@gmail.com>
This RPC has never been used, and is now superceded by GetMempoolStream.
Instead of the Sleep function pointer being a standalong global
variable, move it into a new Time struct, and add a Now function
pointer, so that time.Now() can be mocked. Time.Now() isn't used yet.
This will be cleaner if we need to mock more time-related functions in
the future.
@LarryRuane
Copy link
Collaborator

Rebased onto the latest master branch, force-pushed.

@LarryRuane LarryRuane merged commit b1f3687 into zcash:master Jul 29, 2021
LarryRuane pushed a commit to LarryRuane/lightwalletd that referenced this pull request Dec 9, 2021
This was removed as part of PR zcash#358 (commit "remove gRPC GetMempoolTx")
but should not have been, so it's being restored.
LarryRuane pushed a commit that referenced this pull request Dec 9, 2021
This was removed as part of PR #358 (commit "remove gRPC GetMempoolTx")
but should not have been, so it's being restored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants