The Web 3.0 library lip2p (GO and Rust implementations are by far the most used), is a key modular set of networking tools for building peer-to-peer applications. Knowing what it does and when to used it, will save us of a lot of pain in our technical decisions and it will take care of our bank account. Web 3.0 represents a marked shift from a centralized ownership and control to a decentralized one and mistakes will be made. The important point is to never lose focus on the business model based in decentralized networks with creators, organizations, communities and users of content and data to their system with incentives, such as digital assets. That incentivize the users to become a part of the network and to participate by contributing to the network in some way and make "their living out of it".
Any technical solution or decisions should have this goal in mind and knowing our stuff helps in a long way to achieve this.
Peer-to-peer networks have many advantages over the old client/server model, there are also challenges. In the process of overcoming these challenges, while building IPFS, the contributors of the IPFS project took care of building solutions in a modular, composable way, namely libp2p. Despite libp2p being the networking layer of IPFS, it does not require or depend on IPFS, and today many projects (like Polkadot) use libp2p as their network transport layer.
Every libp2p peer is uniquely identified by their Peer ID, which is derived from a private cryptographic key. Peer ids and their corresponding keys allow us to authenticate remote peers, so that we can be sure we’re talking to the correct peer and not an imposter.
libp2p does not provide an authorization framework “out of the box”. To design an authorization system on libp2p, we need to rely on the authentication of peer ids and build an association between peer ids and permissions, with the Peer ID serving the same function as the “username” in traditional authorization frameworks, and the peer’s private key serving as the “password”. This allow us to reject requests from untrusted peers.
A Noise Protocol begins with two parties exchanging handshake messages. During this handshake phase the parties exchange DH(Diffie-Hellman) public keys and perform a sequence of DH(Diffie-Hellman) operations, hashing the DH(Diffie-Hellman) results into a shared secret key. After the handshake phase each party can use this shared key to send encrypted transport messages.
One of libp2p’s core requirements is to be transport agnostic. This means that the decision of what transport protocol to use is up to the us, and an application can support many different transports at the same time.
- Standalone Node Connectivity
- TCP (Standalone ⇄ Standalone)
- QUIC-UDP (Standalone ⇄ Standalone)
- Hole Punching (Public Node ⇄ Private Node)
- Browser Node Connectivity
The Swarm struct contains all active and pending connections to remotes and manages the state of all the substreams that have been opened, and all the upgrades that were built upon these substreams.
A Swarm requires three things:
- Identity of the local node (PeerId).
- An implementation of the Transport trait.
- An implementation of the NetworkBehaviour trait.
The libp2p library enables many discovery mechanisms or even write our own. These are the two main ones:
- Rendezvous Protocol, lightweight mechanism for generalized peer discovery.
- Identify - Kademlia, Identify protocol using Peer Routing (is the process of discovering peer addresses by using the knowledge of other peers) with DHT(Distributed Hash Table) implementation based on the Kademlia for storing those peer addresses.
Note: Mechanisms like bootstrap or mDNS are limited. In the case of bootstrapping, only connect to the node listed and stops and in case of mDNS only connect to nodes found on the same LAN (local area network).
A basic p2p application implementation using libp2p could be found in here basic-p2p.
A IPFS private network was created with two nodes. A private IPFS network allows only to connect to other peers who have a shared secret key. Each node will become part of the IPFS bootstrap list (is a list of peers with which the IPFS daemon learns about other peers on the network). Nodes in that network don't respond to communications from nodes outside that network.
The Rust project peer-identity was used to retrieve the peer information using the Identify protocol.
Received info from peer Id 12D3KooWDfaWHmKi9XgrDw6e4tgu3noyjm8DmLRuQwyqVszdbcAe (node-a)
{
public_key: Ed25519(PublicKey(compressed): eeb1fd8de179b6239867eb915496e6444feee9188fcaa7d9bf35f6dd7b2e20),
protocol_version: "ipfs/0.1.0", agent_version: "kubo/0.21.0-dev/78895a1/docker",
listen_addrs: ["/ip4/10.244.0.10/tcp/4001", "/ip4/10.244.0.10/udp/4001/quic", "/ip4/10.244.0.10/udp/4001/quic-v1"],
protocols: ["/ipfs/ping/1.0.0", "/libp2p/circuit/relay/0.2.0/stop", "/ipfs/lan/kad/1.0.0", "/libp2p/autonat/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/bitswap/1.2.0", "/ipfs/bitswap/1.1.0", "/ipfs/bitswap/1.0.0", "/ipfs/bitswap", "/x/"],
observed_addr: "/ip4/10.244.0.1/tcp/45287"
}Geting statistics about the node-a DHT(s).
kubectl exec -it ipfs-node-a --namespace=overlay-network -- sh
ipfs stats dhtProtocols:
- DHT WAN:/ipfs/kad/1.0.0
- DHT LAN:/ipfs/lan/kad/1.0.0
Received info from peer Id 12D3KooWRt8ciG9Bz2BpjKV3416fWcAvRPchGWXt6jNbaoTfJGto (node-b)
{
public_key: Ed25519(PublicKey(compressed): 392e784b87befffcadaf4644fed53d215f7453d6da2a479acecd94af59b7ccb),
protocol_version: "ipfs/0.1.0", agent_version: "kubo/0.21.0-dev/78895a1/docker",
listen_addrs: ["/ip4/10.244.0.8/tcp/4001", "/ip4/10.244.0.8/udp/4001/quic", "/ip4/10.244.0.8/udp/4001/quic-v1"],
protocols: ["/ipfs/ping/1.0.0", "/libp2p/circuit/relay/0.2.0/stop", "/ipfs/lan/kad/1.0.0", "/libp2p/autonat/1.0.0", "/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/bitswap/1.2.0", "/ipfs/bitswap/1.1.0", "/ipfs/bitswap/1.0.0", "/ipfs/bitswap", "/x/"],
observed_addr: "/ip4/10.244.0.1/tcp/5686"
}Geting statistics about the node-b DHT(s).
kubectl exec -it ipfs-node-b --namespace=overlay-network -- sh
ipfs stats dhtProtocols:
- DHT WAN:/ipfs/kad/1.0.0
- DHT LAN:/ipfs/lan/kad/1.0.0
The Identify protocol must be manually hooked up to Kademlia through calls to Kademlia::add_address. If we choose not to use the Identify protocol, and do not provide an alternative peer discovery mechanism, a Kademlia node will not discover nodes beyond the network's boot nodes. Without the Identify protocol, existing nodes in the kademlia network cannot obtain the listen addresses of nodes querying them, and thus will not be able to add them to their routing table.(Source).
| DHT | Protocol | Discovery Mechanism |
|---|---|---|
| WAN | /ipfs/kad/1.0.0 | Identify protocol |
| LAN | /ipfs/lan/kad/1.0.0 | Multicast DNS (mDNS) |
Note: This is called Dual DHT.
A libp2p private network was created with four nodes. Each node will have the Identify("/ipfs/id/1.0.0") and Kademlia("/ipfs/kad/1.0.0") behaviour configure. A new node (discovery-Identify-kademlia) will be added to the libp2p private network and it will have to discover every other node in the network just by connecting to node-d using the Indentify+Kademlia behaviour.
Output from the new-node
PeerId: PeerId("12D3KooWLnyhf17V15nSPfJF77opdUbdGx2Q9zK7C48qucwuzgC9")
node-d added.
Listening on "/ip4/127.0.0.1/tcp/4001"
Listening on "/ip4/10.244.0.7/tcp/4001"
Add node
PeerId: PeerId("12D3KooWSAj4PDGEUpywoe7FLcf6ancJmi3AEqACPwxDwZs3zW5g")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.6/tcp/4001
Add node
PeerId: PeerId("12D3KooWSAj4PDGEUpywoe7FLcf6ancJmi3AEqACPwxDwZs3zW5g")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.6/tcp/4001
Add node
PeerId: PeerId("12D3KooWHh541fxK9mJsLxt8wX8cSCfzRsDrKTQaB8EG7R3RYj7z")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.4/tcp/4001
Add node
PeerId: PeerId("12D3KooWHh541fxK9mJsLxt8wX8cSCfzRsDrKTQaB8EG7R3RYj7z")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.4/tcp/4001
Add node
PeerId: PeerId("12D3KooWAXY6cACWiab9uM4ss4Uas3Y6RwK5J3msFCvbMaZfcKaV")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.3/tcp/4001
Add node
PeerId: PeerId("12D3KooWAXY6cACWiab9uM4ss4Uas3Y6RwK5J3msFCvbMaZfcKaV")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.3/tcp/4001
Add node
PeerId: PeerId("12D3KooWJXMpHfCRtddGzZuN4z5Za3iAbikPt5Wav9vRUAxKzdEQ")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.5/tcp/4001
Add node
PeerId: PeerId("12D3KooWJXMpHfCRtddGzZuN4z5Za3iAbikPt5Wav9vRUAxKzdEQ")
Protocols: ["/ipfs/id/1.0.0", "/ipfs/id/push/1.0.0", "/ipfs/kad/1.0.0"]
Address /ip4/10.244.0.5/tcp/4001Note: The Rust project create-keypair will create the identity keypair, the corresponding identifiers peers for the four nodes and save it in files(keypair.bin and peer_id.bin).
The concepts used in the stacks InterPlanetary File System and libp2p are the cornerstone for any Bockchain Architect. Knowing how to use it and when to use it will be essential when we are building software for the token economy.
References:
Distributed Hash Tables (DHTs)
Central repository for work on libp2p
Security Considerations