discv5 topic discovery: efficiency vs resilience tradeoff

[kaiserd]

Discv5 topic discovery uses topic radius as a compromise between efficiency and resilience against eclipse attacks.
I was wondering if topic providers could gain both query efficiency and resilience by placing advertisements at several nodes in ranges of increasing size.
Maybe it would be sufficient to do this without estimating an optimal radius.
Topic providers could start these advertisements in parallel.

Further, topic providers could additionally advertise respective topics as extra fields in their ENRs. This corresponds to advertising topics at radius $2^256$ basically for free, adding additional resilience.

[fjl]

Hi,

the topic system is still in the design phase, and a lot of research has happened since the text that's in the spec now was written. In our current research prototype, the concept of 'topic radius' has been removed. I'm in the last stages of completing the prototype and will update the spec soon.

If you are in interested, you can check out our research at

• https://github.com/harnen/service-discovery-paper
• https://github.com/datahop/p2p-service-discovery

[kaiserd]

Thank you very much :)! I will read the documents you provided and come back.

[kaiserd]

The topic tables proposed in the paper are very similar to what I suggested. So, the problem I described is already covered :). Thank you :).
Waku v2 could be mentioned as one of the applications benefiting from the enhanced discv5 proposed in the paper; this could further strengthen the motivation.
I will write a PR with things that came to my mind while reading the paper (and a few typo fixes etc); maybe this is helpful.

One more question @fjl regarding discovery efficiency:
What is your suggestion for integrating resource restricted devices (e.g. Browser apps, mobile devices) into the discovery process?
Do you envision them being part of discv5 maintaining a routing table or rather just directly query more stable nodes participating in discv5?
(We are planning to specify a peer-exchange protocol that allows restricted devices to retrieve random node sets from nodes actively participating in discv5. Resource restricted devices learn about nodes actively participating in discv5 via DNS discovery.
As opposed to directly using nodes learned via DNS as entry points to the target protocol (e.g. Waku relay), asking them for random node sets which are in turn retrieved via discv5 helps load balancing.)

[fjl]

Regarding the paper, we are still working on it.

When you ask about browsers, what do you have in mind specifically? Browsers cannot participate in discv5 because they cannot speak the UDP protocol. Browsers usually don't have anything to offer to the network, they can only leech, and if they would advertise services, the service availability would be short-lived. For these reasons, I am not interested in adding browser support in this system.

It's a bit different with mobile devices. I think it is possible to run discv5 directly on mobile phones, the protocol is not that resource-hungry.

One thing that I'm interested in adding is explicit support for NAT traversal. Right now, nodes using discv5 can learn about their own internet-facing IP/port using the protocol, but if the NAT is not full-cone, others will not be able to connect to it. It would be very nice if the protocol also provided a way to pierce the NAT in this case. I'm not open to adding packet relay though. From your GitHub activity, I can see that you have worked in this area before. Maybe this topic could be interesting for you to work on?

I'll close this issue now because the original question is kind of answered. Feel free to create another issue for the next topic or ask more questions here.

[kaiserd]

Yes. Thank you. The issue is covered by the new research :).

Regarding browsers, I was thinking about portal, and this discv5 browser implementation.

Yes, I did some NAT related stuff some time ago :). I will have a look and put this as an open issue in our roadmap.
(most of the things (not NAT related) I worked on are on internal University gitlab instances though 😅).

I opened a PR with some comments on the paper's git repo.
As I said there, I could add an estimation of the cost (in expected overlay hops) of random-walk discovery in relation to topic and network size.
We would like to use the new topic discovery for Waku v2 in the future :).