Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running Bitswap Retrieval as a separate process #441

Closed
dirkmc opened this issue Apr 13, 2022 · 1 comment
Closed

Running Bitswap Retrieval as a separate process #441

dirkmc opened this issue Apr 13, 2022 · 1 comment

Comments

@dirkmc
Copy link
Contributor

dirkmc commented Apr 13, 2022

In order to run Bitswap retrieval as a separate process, we would need to

1. Expose a read-only Blockstore interface on Boost

  • The Bitswap server needs to call blockstore methods to respond to incoming requests.
  • We should expose the following Blockstore methods as API endpoints on Boost: Has(cid), Get(cid), GetSize(cid)
  • The DAG store exposes a Blockstore interface over all cids in all deals, so we can connect the API endpoints to this DAG store interface.

2. Make announcements to the index provider each time a deal is added to Boost

  • The Bitswap process will have a libp2p node.
  • We should create an index announcer with the libp2p node's host. The index announcer will announce that this host has the blocks.
  • The Bitswap process can poll Boost for new deals and announce the deals using the index announcer.
  • We should expose an API endpoint on Boost that the Bitswap process can poll for new deals.
  • There was a problem before where the index announcer looked up the address of the storage provider on chain instead of reading the address from the announcement. Need to check if this was fixed.

3. Respond to requests from the indexer node for indices in a piece

  • The Bitswap process will run the indexer announcer.
  • The index announcer takes a callback parameter to query for all cids in a piece
  • We should expose an API endpoint on Boost that serves all cids in a piece

This work would build on top of Create a PoC of bitswap retrieval from Boost

@jacobheun
Copy link
Contributor

This is now being tracked in sub issues of #709.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

2 participants