Saving previously seen nodes for later bootstrapping

[dsvi]

Version information:

N/A

Type:

Enhancement

Severity:

Medium

Description:

Bootstrap nodes built into ipfs client don't seem to work for me at all. Had to find and add some manually.
Generally having so centralized bootstrap system for otherwise decentralized network is its weakness:
#3908
Should ipfs automatically save all the nodes it seen and the time of last seen, to clean out the list later?
That should make the bootstrapping more reliable and truly decentralized.

[whyrusleeping]

Yes! definitely. The blocker here is making sure we store peerstore data to a persistent datastore (right now all peer info is kept in memory).

[djdv]

Related anecdote
Perfect Dark relies on users to manually add nodes, the client retains peers like you had mentioned. It seems to work well for their network/peers. You add a peer to connect to, the client will exchange peer lists with them and add all of them to a ledger, on the next bootstrap the client uses that ledger, it attempts to establish a connection with everyone in it, after enough connection failures peers are removed from the ledger while successful ones remain, this keeps it from filling up with too many dead nodes.

Not sure what approach would be good for ipfs in terms of storing peers and managing stored peers for bootstrap, or what other systems have tried.

[whyrusleeping]

@djdv that sounds like a great approach. Especially tracking failures for removal.

[makew0rld]

So did this happen? Thanks.

[makew0rld]

@Kubuxu @whyrusleeping
What's the state of this?

[Stebalien]

@bigs is working on storing the peerstore on disk (libp2p/go-libp2p-peerstore#28). After that, it'll be a matter of remembering which peers tend to reliably be online.

[bigs]

yup. we'll also need to consider how we re-initialize our TTLs after rebooting, but this is coming down the pike.

[lidel]

Ideas being discussed in libp2p/go-libp2p-kad-dht#254

[blurHY]

GFW blocked all default bootstrap nodes

GFW is the national firewall of China

[Geo25rey]

@whyrusleeping Do you know the status of this issue?

[aschmahmann]

@Geo25rey there are a number of the issues (and some PRs) in the DHT around this that you can checkout.

The short version is that there's interest in doing this and some good proposals, but the plan is not to do it until we've landed higher priority work on performing smooth upgrades to the DHT protocol (e.g. libp2p/go-libp2p-kad-dht#616).

[Geo25rey]

we've landed higher priority work on performing smooth upgrades to the DHT protocol

@aschmahmann What do you mean by that?

[lidel]

Two years later, neither libp2p/go-libp2p-kad-dht#616 nor libp2p/go-libp2p-kad-dht#254 happened.

Meanwhile, various countries and companies can cripple connectivity by blocking well-known list of hardcoded bootstrappers.

I propose we do something rather than keeping the current broken state.
Does not have to be perfect, MVP could live purely in go-ipfs code.
When upstream libs support persistence and fancy logic, we can switch to them.

Help wanted

If someone opens a PR that is preserving currently connected peers (ipfs swarm connect) across restarts, I'm happy to review it.

MVP:

• persist the list of connected peers every 15m / swarm peers is enumerated / on shutdown
• extend bootstrap.go logic to re-connect to persisted peers
  • grab some number of peers that are in your routing table that you're connected to and use them as additional temporary bootstrappers, but don't add them with peerstore.PermanentAddrTTL - these should not be permanent bootstrappers
    • instead, use peerstore.TempAddrTTL or peerstore.ProviderAddrTTL to ensure we keep them around for a short period of time

[schomatis]

Per @lidel, taking this one since no one in the community has tackled it yet (if someone is still interested I can be the guide and reviewer but should say so now).

[schomatis]

Actively working on this. It's taking some time to do it right, will push something by EOW.

[schomatis]

Beautiful people of the ipfsphere, I have a very WIP PR with an initial implementation of this in #8856. Any feedback will be very useful to better understand the use cases we should support.

[ElianaTroper]

One thing that might be useful considering for this (and potentially other cases) is generalizing a bit to have bootstrapping/peers grouped. If I'm shipping an application it might make sense to have 3 groups of bootstrapping nodes - the default nodes to connect to the broader decentralized IPFS ecosystem, a set of nodes that the application shipper is running to enable closer connections to other users that are also using the app (and to take some load off of the default nodes), and previously seen nodes. I could see this being useful with a combo of priority levels and percent of nodes to connect to within a group (e.g. 100% of application shipper nodes at priority 1, 25% of default nodes to reduce the burden on those also at priority 1, and 33% of previously connected nodes at priority 2). I wouldn't want to sample randomly among all nodes in the bootstrapping list, I would want to sample within these groups to maintain better performance of my application.

I don't think this improvement should hold up the initial code, but this would be a more general approach that would solve this PR and would more broadly help IPFS in other use cases too.

Additionally, having nodes grouped could return useful feedback in the future (e.g. all nodes in the default bootstrapping list are not connecting - maybe there's an issue occurring there or they're blocked by e.g. GFW or corporate firewall) and this could trigger further actions.