Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test peer discovery with public DHT #96

Closed
wants to merge 2 commits into from
Closed

Test peer discovery with public DHT #96

wants to merge 2 commits into from

Conversation

2color
Copy link
Collaborator

@2color 2color commented Sep 27, 2023

  • js-peer: connect to public libp2p bootstrap nodes
  • remove dht protocol prefix to use public dht

@vercel
Copy link

vercel bot commented Sep 27, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
universal-connectivity ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 27, 2023 11:54am

@@ -72,7 +74,7 @@ export async function startLibp2p() {
ignoreDuplicatePublishError: true,
}),
dht: kadDHT({
protocolPrefix: "/universal-connectivity",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need to be changed in all clients if they should be compatible. I am not sure if we want to do this though.

Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project. I don't think we want that? Not until we change it to be a "generic peer routing" DHT I guess?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mostly agree. The key question I'm trying to answer with this PR, is how well ambient peer discovery and routing work for js peers in this app (in the browser) using the public bootstrappers and how that compares with the approach of a single rust peer, single go peer and multiple browser js peers.

I haven't found a definitive answer to this in any of the libp2p specs or the docs.

Intuitively, it makes sense to me that all peers in this app use a separate DHT. But if we assume that we mostly have 50 concurrent browser peers with no DHT server and a single rust node, how well would all peers be able to propagate their messages to the other peers?

This is clearly a question that this app should be able to answer. And the tradeoffs are important for any application built with libp2p targeting the web platform.

Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project.

This is default behaviour of the go kad-dht implementation, https://github.com/libp2p/go-libp2p-kad-dht/blob/master/internal/config/config.go#L19-L20

I didn't see it explicitly encouraged or discouraged and it seems like it wouldn't be the default if that were the case

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somebody coming to libp2p via this project would get the impression that they can just use the IPFS DHT for their project.

This is default behaviour of the go kad-dht implementation, libp2p/go-libp2p-kad-dht@master/internal/config/config.go#L19-L20

I didn't see it explicitly encouraged or discouraged and it seems like it wouldn't be the default if that were the case

It is also the default for rust-libp2p actually :)

Intuitively, it makes sense to me that all peers in this app use a separate DHT. But if we assume that we mostly have 50 concurrent browser peers with no DHT server and a single rust node, how well would all peers be able to propagate their messages to the other peers?

This is clearly a question that this app should be able to answer. And the tradeoffs are important for any application built with libp2p targeting the web platform.

100%! My thoughts are:

  • The web-app MUST provide the ability to just paste a server-node address and connect to it.
  • We SHOULD provide a node that users can connect to. Maybe not automatically but pre-fill it in the UI or make a separate button? What would be good to teach here is that by no means is this particular peer special.
  • Server nodes SHOULD support the relay protocol to allow browsers to establish WebRTC connections to each other.
  • Server nodes SHOULD form their own DHT, separate from IPFS
  • Server nodes SHOULD support gossipsub

Imagine this:

  • We both spin up our own bootstrap nodes in the cloud, completely oblivious that there is already one, i.e. we don't connect it directly to the provided one.
  • We both start the web app and connect to our own bootstrap node by pasting the address but also connect to the provided bootstrap node.
  • Now we can technically discover each other via gossipsub or something else.
  • We use the relay to hole-punch a browser-to-browser WebRTC connection.
  • It would be amazing if somehow, our two personal server nodes would learn about each other. Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • It would be amazing if somehow, our two personal server nodes would learn about each other. Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?

If you tell me about your bootstrap node, I can connect to it. My bootstrap node could then ask me, who I am connected to and find out about your bootstrap node. They can connect to each other, discover that they are both server nodes and form a DHT.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Perhaps we need a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?

We would definitely need base this on peer records and not just plain addresses. It is still problematic because you could just make up a new identity, put a bunch of addresses in a peer record, distribute them and thus trigger dials to these addresses which is essentially an amplification attack :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a simple "ambient-peer" request-response protocol that goes something like: "Send me all peers you are connected to"?

That's funny, we were just talking about something like this at the js-colo. We used to have something similar in webrtc-star so there is prior art - the signalling server used for the initial SDP handshake also functioned as a peer discovery method since when you connected to it, it would send you a list of other peers connected to the server.

These are high-value peers for browser nodes since they are guaranteed to support the same transports as you.

The thought was, we could have a protocol that supports returning a list of peers you know about filtered by some sort of attribute, the initial one would be the types of multiaddr they are available on - since there's little use returning tcp/quick-only peers to browser nodes.

Copy link
Contributor

@thomaseizinger thomaseizinger Sep 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very interesting! Thanks for sharing :)

But, filtering or not, we'd have to figure out how we can trust the peer we just connected to with the data they've given us.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote up a proposal here: libp2p/specs#587

@2color 2color closed this Apr 29, 2024
@2color 2color deleted the public-boostraps branch May 2, 2024 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants