Closest peers query is not efficient

[vasco-santos]

By default, dht.getClosestPeers is crawling all the network to find the closestPeers, which takes a long time and hits heavily the CPU when we have a large number of peers.

This was also a problem observed in #85 , as this took a lot of time and returned a large set of peers, which ended up failing the dht.put for lack of response of several peers. Moreover, queries also have an impact on random-walk performance.

We need a deterministic way of ending this network query, as well as to refactor closestPeers and put to be more fault tolerant.

We should stream the query and have an abort logic that would allow us to stop the query at a certain point. In addition, we need to have a smarter connection manager to full enable this.

cc @jacobheun @dirkmc

[vasco-santos]

We should consider: libp2p/go-libp2p-kad-dht#237

[dirkmc]

SGTM 👍

I'm currently implementing streaming as part of the async/await refactor

Should we simply end the query when the timeout occurs? I've also implemented this behaviour in the refactor.

[vasco-santos]

We currently do not have a timeout for put / getClosestPeers, but it can be a possibility.

In addition, I think we should define a minimum number of peers (from the closest) to put, instead of needing by default to put all of them (I am against this, but while we don't have a proper logic to identify if a peer has a dht running, we should maybe do that).

Also, if we can define in the closestPeers an option for the numberOfPeers / minNumberOfPeers, it can also help. However, I am not sure what should be the ideal values to use.

Ideally, we should not need to do this kind of stuff, and if we do, we should open issues for guaranteeing that when we are in a better position regarding performance and smarter connection manager, we can aim to go back to "normal"

[jacobheun]

but while we don't have a proper logic to identify if a peer has a dht running, we should maybe do that

We're working on support for this for the Gossipsub implementation, it should hopefully be added to the switch soon (mostly just testing and interop testing left). This would enable us to check peerInfo.protocols for the dht protocol, so we could prevent actions to those peers who are missing it.