Skip to content
This repository has been archived by the owner on Oct 23, 2022. It is now read-only.

Initial content discovery #260

Merged
merged 6 commits into from
Jul 28, 2020
Merged

Conversation

ljedrz
Copy link
Member

@ljedrz ljedrz commented Jul 23, 2020

An initial implementation of content discovery via DHT, where a get_block request that doesn't succeed locally results in a call to Kademlia::get_providers. It also fixes (AKA fixes in most cases) the previously prototyped display of Kademlia content keys, in this case Cids.

cc #10

@ljedrz ljedrz requested a review from koivunej July 23, 2020 13:18
@ljedrz ljedrz changed the title Actual content discovery Initial content discovery Jul 23, 2020
tests/kademlia.rs Outdated Show resolved Hide resolved
Signed-off-by: ljedrz <ljedrz@gmail.com>
…ry one

Signed-off-by: ljedrz <ljedrz@gmail.com>
Signed-off-by: ljedrz <ljedrz@gmail.com>
Signed-off-by: ljedrz <ljedrz@gmail.com>
Signed-off-by: ljedrz <ljedrz@gmail.com>
Signed-off-by: ljedrz <ljedrz@gmail.com>
Comment on lines +431 to 435
// FIXME: it would probably be best if this could return a SubscriptionFuture, so
// that the put_block operation truly finishes only when the block is already being
// provided; it is, however, pretty tricky in terms of internal communication between
// Ipfs and IpfsFuture objects - it would currently require some extra back-and-forth
pub fn provide_block(&mut self, cid: Cid) {
Copy link
Collaborator

@koivunej koivunej Jul 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not 100% with you on this FIXME but no need to change it now. To elaborate:

"Not 100% with you" as in is that the saving blocks should be as decoupled as possible and providing should only happen for some subset of blocks: possibly only the pinned roots, or pinned roots and their subroots or anything more complex would need to be provided for. Where "subroots" would be the roots of the files in a pinned larger directory structure for the invidiual files.

Decoupling providing and put_block because providing can be quite expensive, and we might not want to do them concurrently at all. Additionally not sure if it's a good idea to plan to use start_providing unless we can come up with a really nice backend for the kademlia store. Otherwise we could just use the simpler "push" API and handle the timing ourselves.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough 👍

@koivunej
Copy link
Collaborator

Flaky #134 with the readme-doctest again.

Copy link
Collaborator

@koivunej koivunej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good! Thanks.

bors r+

@bors
Copy link
Contributor

bors bot commented Jul 28, 2020

Build succeeded:

@bors bors bot merged commit 7ebe1a6 into rs-ipfs:master Jul 28, 2020
@ljedrz ljedrz deleted the actual_content_discovery branch July 28, 2020 08:38
@koivunej koivunej mentioned this pull request Sep 22, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants