| title |
|---|
-chap-num- Networking |
:::info This chapter, in its current form, is incomplete and considered work in progress. Authors appreciate receiving requests for clarification or any reports regarding deviation from the current Polkadot network protocol. This can be done by filing an issue in Polkadot Specification repository. :::
The Polkadot network is decentralized and does not rely on any central authority or entity to achieve its fullest potential of provided functionality. The networking protocol is based on a family of open protocols, including protocol implemented libp2p, e.g., the distributed Kademlia hash table, which is used for peer discovery.
This chapter walks through the behavior of the networking implementation of the Polkadot Host and defines the network messages. The implementation details of the libp2p protocols used are specified in external sources as described in Section -sec-num-ref-
Complete specification of the Polkadot networking protocol relies on the following external protocols:
-
libp2p - libp2p is a modular peer-to-peer networking stack composed of many modules and different parts. includes the multiplexing protocols and
-
libp2p addressing - The Polkadot Host uses the libp2p addressing system to identify and connect to peers.
-
Kademlia - Kademlia is a distributed hash table for decentralized peer-to-peer networks. The Polkadot Host uses Kademlia for peer discovery.
-
Noise - The Noise protocol is a framework for building cryptographic protocols. The Polkadot Host uses Noise to establish the encryption layer to remote peers.
-
yamux - yamux is a multiplexing protocol developed by HashiCorp. It is the de-facto standard for the Polkadot Host. Section -sec-num-ref- describes the subprotocol in more detail.
-
Protocol Buffers - Protocol Buffers is a language-neutral, platform-neutral mechanism for serializing structured data and is developed by Google. The Polkadot Host uses Protocol Buffers to serialize specific messages, as clarified in Section -sec-num-ref-.
Like any other distributed system, each Polkadot Host node has a unique global identifier. This identifier, called PeerId, serves as a singular reference to a particular node within the overall network.
In Polkadot, each node is required to maintain its own pair of ED25519 cryptographic keys from which the PeerId is derived.
The Polkadot node’s PeerId is structured based on the libp2p specification but does not fully conform to the specification. In particular, it does not support CID and the only supported key type is ED25519. The PeerId is built by hashing the encoded public key with multihash and represented as follows:
:::definition
The byte representation of the PeerId is always of the following bytes in this exact order:
$$ {b}{{0}}={0} $$ $$ {b}{{1}}={36} $$ $$ {b}{{2}}={8} $$ $$ {b}{{3}}={1} $$ $$ {b}{{4}}={18} $$ $$ {b}{{5}}={32} $$ $$ {b}_{{{6}.{.37}}}=\ldots $$
where
-
${b}_{{0}}$ is the multihash prefix of value${0}$ (implying no hashing is used). -
${b}_{{1}}$ the length of the PeerId (remaining bytes). -
${b}{{2}}$ and ${b}{{3}}$ are a protobuf encoded field-value pair indicating the used key type (field
${1}$ of value${1}$ implies ED25519). -
${b}{{4}}$, ${b}{{5}}$ and ${b}{{{6}.{.37}}}$ are a protobuf encoded field-value pair where ${b}{{5}}$ indicates the length of the public key followed by the raw ED25519 public key itself, which varies for each Polkadot Host and is always 32 bytes (field
${2}$ contains the public key, which has a field value length prefix).
:::
The Polkadot Host uses various mechanisms to find peers within the network, to establish and maintain a list of peers, and to share that list with other peers from the network as follows:
-
Bootstrap nodes are hard-coded node identities and addresses provided by the genesis state (Section -sec-num-ref-).
-
mDNS is a protocol that performs a broadcast to the local network. Nodes that might be listening can respond to the broadcast. The libp2p mDNS specification defines this process in more detail. This protocol is an optional implementation detail for Polkadot Host implementers and is not required to participate in the Polkadot network.
-
Kademlia requests invoking Kademlia requests, where nodes respond with their list of available peers. Kademlia requests are performed on a specific substream as described in Section -sec-num-ref-.
Polkadot nodes connect to peers by establishing a TCP connection. Once established, the node initiates a handshake with the remote peers on the encryption layer. An additional layer on top of the encryption layer, known as the multiplexing layer, allows a connection to be split into substreams, as described by the yamux specification, either by the local or remote node.
The Polkadot node supports two types of substream protocols. Section -sec-num-ref- describes the usage of each type in more detail:
-
Request-Response substreams: After the protocol is negotiated by the multiplexing layer, the initiator sends a single message containing a request. The responder then sends a response, after which the substream is then immediately closed. The requests and responses are prefixed with their LEB128 encoded length.
-
Notification substreams. After the protocol is negotiated, the initiator sends a single handshake message. The responder can then either accept the substream by sending its own handshake or reject it by closing the substream. After the substream has been accepted, the initiator can send an unbound number of individual messages. The responder keeps its sending side of the substream open, despite not sending anything anymore, and can later close it in order to signal to the initiator that it no longer wishes to communicate.
Handshakes and messages are prefixed with their LEB128 encoded lengths. A handshake can be empty, in which case the length prefix would be 0.
Connections are established by using the following protocols:
-
/noise- a protocol that is announced when a connection to a peer is established. -
/multistream/1.0.0- a protocol that is announced when negotiating an encryption protocol or a substream. -
/yamux/1.0.0- a protocol used during yamux negotiation. See Section -sec-num-ref- for more information.
The Polkadot Host can establish a connection with any peer of which it knows the address. The Polkadot Host supports multiple networking protocols:
-
TCP/IP with addresses in the form of
/ip4/1.2.3.4/tcp/30333to establish a TCP connection and negotiate encryption and a multiplexing layer. -
WebSocket with addresses in the form of
/ip4/1.2.3.4/tcp/30333/wsto establish a TCP connection and negotiate the WebSocket protocol within the connection. Additionally, the encryption and multiplexing layer are negotiated within the WebSocket connection. -
DNS addresses in form of
/dns/example.com/tcp/30333and/dns/example.com/tcp/30333/ws.
The addressing system is described in the libp2p addressing specification. After a base-layer protocol is established, the Polkadot Host will apply the Noise protocol to establish the encryption layer as described in Section -sec-num-ref-.
Polkadot protocol uses the libp2p Noise framework to build an encryption protocol. The Noise protocol is a framework for building encryption protocols. libp2p utilizes that protocol for establishing encrypted communication channels. Refer to the libp2p Secure Channel Handshake specification for a detailed description.
Polkadot nodes use the XX handshake pattern to establish a connection between peers. The three following steps are required to complete the handshake process:
-
The initiator generates a key pair and sends the public key to the responder. The Noise specification and the libp2p PeerId specification describe keypairs in more detail.
-
The responder generates its own key pair and sends its public key back to the initiator. After that, the responder derives a shared secret and uses it to encrypt all further communication. The responder now sends its static Noise public key (which may change anytime and does not need to be persisted on disk), its libp2p public key, and a signature of the static Noise public key signed with the libp2p public key.
-
The initiator derives a shared secret and uses it to encrypt all further communication. It also sends its static Noise public key, libp2p public key, and signature to the responder.
After these three steps, both the initiator and responder derive a new shared secret using the static and session-defined Noise keys, which are used to encrypt all further communication.
After the node establishes a connection with a peer, the use of multiplexing allows the Polkadot Host to open substreams. libp2p uses the yamux protocol to manage substreams and to allow the negotiation of application-specific protocols, where each protocol serves a specific utility.
The Polkadot Host uses multiple substreams whose usage depends on a specific purpose. Each substream is either a Request-Response substream or a Notification substream, as described in Section -sec-num-ref-.
:::info
The prefixes on those substreams are known as protocol identifiers and are used to segregate communications to specific networks. This prevents any interference with other networks. dot is used exclusively for Polkadot. Kusama, for example, uses the protocol identifier ksmcc3.
:::
-
/ipfs/ping/1.0.0- Open a standardized substream libp2p to a peer and initialize a ping to verify if a connection is still alive. If the peer does not respond, the connection is dropped. This is a Request-Response substream.Further specification and reference implementations are available in the libp2p documentation.
-
/ipfs/id/1.0.0- Open a standardized libp2p substream to a peer to ask for information about that peer. This is a Request-Response substream, but the initiator does not send any message to the responder and only waits for the response.Further specification and reference implementations are available in the libp2p documentation.
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/kad- Open a standardized substream for KademliaFIND_NODErequests. This is a Request-Response substream, as defined by the libp2p standard.Further specification and reference implementation are available on Wikipedia respectively the golang Github repository.
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/light/2- a request and response protocol that allows a light client to request information about the state. This is a Request-Response substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /dot/light/2 is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/block-announces/1- a substream/notification protocol which sends blocks to connected peers. This is a Notification substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /dot/block-announces/1 is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/sync/2- a request and response protocol that allows the Polkadot Host to request information about blocks. This is a Request-Response substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /dot/sync/2 is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/sync/warp- a request and response protocol that allows the Polkadot Host to perform a warp sync request. This is a Request-Response substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /dot/sync/warp is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/transactions/1- a substream/notification protocol which sends transactions to connected peers. This is a Notification substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /dot/transactions/1 is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/grandpa/1- a substream/notification protocol that sends GRANDPA votes to connected peers. This is a Notification substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /paritytech/grandpa/1 is also a valid substream for those messages.
:::
-
/91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/beefy/1- a substream/notification protocol which sends signed BEEFY payloads, as described in Section -sec-num-ref-, to connected peers. This is a Notification substream.The messages are specified in Section -sec-num-ref-.
:::info
For backward compatibility reasons, /paritytech/beefy/1 is also a valid substream for those messages.
:::
The Polkadot Host must actively communicate with the network in order to participate in the validation process or act as a full node.
:::info The Polkadot network originally only used SCALE encoding for all message formats. Meanwhile, Protobuf has been adopted for certain messages. The encoding of each listed message is always SCALE encoded unless Protobuf is explicitly mentioned. Encoding and message formats are subject to change. :::
The discovery mechanism enables Polkadot nodes to both publish their local addresses and learn about other nodes' identifiers and addresses. The Authority discovery mechanism differs from the bootstrap mechanism, described in Section -sec-num-ref-, in that it restricts the discovery output to nodes currently holding the authority role (e.g., validators).
The following requests are exposed by the discovery authority to Polkadot nodes.
:::definition
An authority addresses request is a request that Polkadot nodes can send to the authority discovery mechanism in order to request the addresses of an authority node. The request has the following format:
where
-
$\texttt{authorityId}$ is theauthorityId256-bit identifier representing the public key of the targeted authority node.
expected response
The response to the previous query includes an enum with one of the following values:
| Value | Description |
|---|---|
| None | A type representing no value |
| HashSet[Multiaddr] | An unordered collection of unique Multiaddr elements |
with
Multiaddra Multiaddr data structure. :::
:::definition
An authority identifier request is a request that Polkadot nodes can send to the authority discovery mechanism in order to request the AuthorityId of an authority node. The request has the following format:
where
-
$\texttt{PeerId}$ is the Polkadot node’s PeerId (Definition -def-num-ref-).
expected response The response to the previous query includes an enum with one of the following values:
| Value | Description |
|---|---|
| None | A type representing no value |
| HashSet[authorityId] | An unordered collection of unique authorityId elements |
with
authorityIdis the 256-bit identifier representing the public key of the requestedPeerId.
:::
The authority discovery mechanism triggers operations to publish a SignedAuthorityRecord of the addresses of authorities it knows from its current and next authority sets into the DHT. The SignedAuthorityRecord and the publish operation are created as follows:
:::definition
The SignedAuthorityRecord is a Protobuf serialized structure representing the authority records and signature to send over the wire.
It is defined in the following format:
| Type | Id | Description |
|---|---|---|
| AuthorityRecord | 1 | Serialized authority record |
bytes |
2 | An Schnorrkel/Ristretto x25519 ("sr25519") signature |
| PeerSignature | 3 | Serialized peer signature |
where
AuthorityRecord is a serialized Protobuf structure that lists the addresses of authority nodes that are currently part of the authority set.
| Type | Id | Description |
|---|---|---|
repeated bytes |
1 | Binary representation of zero or more multiaddresses through which a node is reachable |
PeerSignature is a Protobuf serialized structure indicating the signature and public key used to sign and verify the AuthorityRecord.
This is the protobuf structure used to exchange the signature with other nodes.
| Type | Id | Description |
|---|---|---|
bytes |
1 | An sr25519 signature |
bytes |
2 | A sr25519 public key used to verify the signature |
:::
:::definition
For each authority node put_value operation that triggers the publishing operation into the DHT with the following format:
$$ \texttt{put_value}{\left(\texttt{KademliaKey}{i} , \texttt{Sig}{AR}\right)} $$
where
-
$\texttt{KademliaKey}_{i}$ is the$Sha256$ hash of the authorityId of node$i$ . -
$\texttt{Sig}_{AR}$ is theSignedAuthorityRecorddescribed above(Definition -def-num-ref-). :::
The authority discovery mechanism also invokes operations on the DHT to discover the addresses of authority nodes, as follows:
:::definition
Periodically, the authority discovery performs a number of get_value operations in the following format:
$$
\texttt{get_value}{\left(\texttt{KademliaKey}_{i}\right)}
$$
where
-
$\texttt{KademliaKey}_{i}$ is the$Sha256$ hash of theauthorityIdof node$i$ selected from the current authority set. :::
When the node creates or receives a new block, it must be announced to the network. Other nodes within the network will track this announcement and can request information about this block. The mechanism for tracking announcements and requesting the required data is implementation-specific.
Block announcements, requests, and responses are sent over the substream as described in Definition -def-num-ref-.
:::definition
The BlockAnnounceHandshake initializes a substream to a remote peer. Once established, all BlockAnounce messages (Definition -def-num-ref-) created by the node are sent to the /dot/block-announces/1 substream.
The BlockAnnounceHandshake is a structure of the following format:
$$ {B}{A}{{h}}=\text{Enc}{{\text{SC}}}{\left({R},{N}{{B}},{h}{{B}},{h}_{{G}}\right)} $$
where
$$ {R}={\left\lbrace\begin{matrix}{1}&\text{The node is a full node}\{2}&\text{The node is a light client}\{4}&\text{The node is a validator}\end{matrix}\right.} $$ $$ {N}{{B}}=\text{Best block number according to the node} $$ $$ {h}{{B}}=\text{Best block hash according to the node} $$ $$ {h}_{{G}}=\text{Genesis block hash according to the node} $$
:::
:::definition
The BlockAnnounce message is sent to the specified substream and indicates to remote peers that the node has either created or received a new block.
The message is structured in the following format:
where
:::
Block requests can be used to retrieve a range of blocks from peers. Those messages are sent over the /dot/sync/2 substream.
:::definition
The BlockRequest message is a Protobuf serialized structure of the following format:
| Type | Id | Description | Value |
|---|---|---|---|
uint32 |
1 | Bits of block data to request | |
oneof |
Start from this block | ||
| Direction | 5 | Sequence direction, interpreted as Id 0 (ascending) if missing. | |
uint32 |
6 | Maximum amount (optional) |
where
-
${B}{{f}}$ indicates all the fields that should be included in the request. its big-endian encoded bitmask that applies to all desired fields with bitwise OR operations. For example, the ${B}{{f}}$ value to request Header and Justification is 0001 0001 (17).
Field Value Header 0000 0001 Body 0000 0010 Justification 0001 0000 -
${B}_{{s}}$ is a Protobuf structure indicating a varying data type (enum) of the following values:Type Id Description bytes2 The block hash bytes3 The block number -
Direction is a Protobuf structure indicating the sequence direction of the requested blocks. The structure is a varying data type (enum) of the following format:
Id Description 0 Enumerate in ascending order (from child to parent) 1 Enumerate in descending order (from parent to canonical child) -
${B}_{{m}}$ is the number of blocks to be returned. An implementation-defined maximum is used when unspecified.
:::
:::definition
The BlockResponse message is received after sending a BlockRequest message to a peer. The message is a Protobuf serialized structure of the following format:
| Type | Id | Description |
|---|---|---|
| Repeated BlockData | 1 | Block data for the requested sequence |
where BlockData is a Protobuf structure containing the requested blocks. Do note that the optional values are either present or absent depending on the requested fields (bitmask value). The structure has the following format:
| Type | Id | Description | Value |
|---|---|---|---|
bytes |
1 | Block header hash | Definition -def-num-ref- |
bytes |
2 | Block header (optional) | Definition -def-num-ref- |
repeated bytes |
3 | Block body (optional) | Definition -def-num-ref- |
bytes |
4 | Block receipt (optional) | |
bytes |
5 | Block message queue (optional) | |
bytes |
6 | Justification (optional) | Definition -def-num-ref- |
bool |
7 | Indicates whether the justification is empty (i.e., should be ignored) |
:::
The Polkadot Host can request the state in the form of a key/value list at a specified block.
When receiving state entries from the state response messages (Definition -def-num-ref-), the node can verify the entries with the entry proof (id 1 in KeyValueStorage) against the Merkle root in the block header (of the block specified in Definition -def-num-ref-). Once the state response message claims that all entries have been sent (id 3 in KeyValueStorage), the node can use all collected entry proofs and validate them against the Merkle root to confirm that claim.
See the synchronization chapter for more information (Chapter -chap-num-ref-).
:::definition
A state request is sent to a peer to request the state at a specified block. The message is a single 32-byte Blake2 hash which indicates the block from which the sync should start.
Depending on what substream is used, the remote peer either sends back a state response (Definition -def-num-ref-) on the /dot/sync/2 substream or a warp sync proof (Definition -def-num-ref-) on the /dot/sync/warp.
:::
:::definition
The state response is sent to the peer that initialized the state request (Definition -def-num-ref-) and contains a list of key/value entries with an associated proof. This response is sent continuously until all key/value pairs have been submitted.
| Type | Id | Description |
|---|---|---|
repeated KeyValueStateEntry |
1 | State entries |
bytes |
2 | State proof |
where KeyValueStateEntry is of the following format:
| Type | Id | Description |
|---|---|---|
bytes |
1 | Root of the entry, empty if top-level |
repeated StateEntry |
2 | Collection of key/values |
bool |
3 | Equal 'true' if there are no more keys to return. |
and StateEntry:
| Type | Id | Description |
|---|---|---|
bytes |
1 | The key of the entry |
bytes |
2 | The value of the entry |
:::
The warp sync protocols allow nodes to retrieve blocks from remote peers where authority set changes occurred. This can be used to speed up synchronization to the latest state.
See the synchronization chapter for more information (Chapter -chap-num-ref-).
:::definition
The warp sync proof message, /dot/sync/warp substream and contains accumulated proof of multiple authority set changes (Section -sec-num-ref-). It’s a data structure of the following format:
$$ {P}={\left({{f}{{x}}\ldots}{{f}{{y}},}{c}\right)} $$
${{f}{{x}}\ldots}{{f}{{y}}}$ is an array consisting of warp sync fragments of the following format:
$$ {{f}{{x}}=}{\left({B}{{h}},{J}^{{{r},\text{stage}}}{\left({B}\right)}\right)} $$
where
:::
Transactions (Section -sec-num-ref-) are sent directly to peers with which the Polkadot Host has an open transaction substream (Definition -def-num-ref-). Polkadot Host implementers should implement a mechanism that only sends a transaction once to each peer and avoids sending duplicates. Sending duplicate transactions might result in undefined consequences such as being blocked for bad behavior by peers.
The mechanism for managing transactions is further described in Section Section -sec-num-ref-.
:::definition
The transactions message is the structure of how the transactions are sent over the network. It is represented by
$$ {M}{{T}}:=\text{Enc}{{\text{SC}}}{\left({C}{{1}},\ldots,{C}{{n}}\right)} $$
in which
$$ {C}{{i}}:=\text{Enc}{{\text{SC}}}{\left({E}_{{i}}\right)}$$
Where each
Transactions are sent over the /dot/transactions/1 substream.
:::
The exchange of GRANDPA messages is conducted on the substream. The process for the creation and distribution of these messages is described in Chapter -chap-num-ref-. The underlying messages are specified in this section.
:::definition
A GRANDPA gossip message,
$$ {M}={\left\lbrace\begin{matrix}{0}&\text{Vote message}&{V}{{m}}\{1}&\text{Commit message}&{C}{{m}}\{2}&\text{Neighbor message}&{N}{{m}}\{3}&\text{Catch-up request message}&{R}{{m}}\{4}&\text{Catch-up message}&{U}_{{m}}\end{matrix}\right.} $$
where
-
${V}_{{m}}$ is defined in Definition -def-num-ref-. -
${C}_{{m}}$ is defined in Definition -def-num-ref-. -
${N}_{{m}}$ is defined in Definition -def-num-ref-. -
${R}_{{m}}$ is defined in Definition -def-num-ref-. -
${U}_{{M}}$ is defined in Definition -def-num-ref-.
:::
:::definition
A GRANDPA vote message by voter
$$ {{M}{{v}}^{{{r},\text{stage}}}}{\left({B}\right)}:=\text{Enc}{{\text{SC}}}{\left({r},\text{id}{{{\mathbb{{V}}}}},\text{SigMsg}\right)} $$ $$ \text{SigMsg}:={\left(\text{msg},{\text{Sig}{{{v}{{i}}}}^{{{r},\text{stage}}}},{v}{{\text{id}}}\right)} $$ $$ \text{msg}:=\text{Enc}{{\text{SC}}}{\left(\text{stage},{{V}{{v}}^{{{r},\text{stage}}}}{\left({B}\right)}\right)} $$
where
-
${r}$ is an unsigned 64-bit integer indicating the Grandpa round number (Definition -def-num-ref-). -
$\text{id}_{{{\mathbb{{V}}}}}$ is an unsigned 64-bit integer indicating the authority Set Id (Definition -def-num-ref-). -
${\text{Sig}{{{v}{{i}}}}^{{{r},\text{stage}}}}$ is a 512-bit byte array containing the signature of the authority (Definition -def-num-ref-).
-
${v}_{{{i}{d}}}$ is a 256-bit byte array containing the ed25519 public key of the authority. -
$\text{stage}$ is a 8-bit integer of value 0 if it’s a pre-vote sub-round, 1 if it’s a pre-commit sub-round or 2 if it’s a primary proposal message. -
${{V}_{{v}}^{{{r},\text{stage}}}}{\left({B}\right)}$ is the GRANDPA vote for block${B}$ (Definition -def-num-ref-).
This message is the sub-component of the GRANDPA gossip message (Definition -def-num-ref-) of type Id 0.
:::
:::definition
The GRANDPA compact justification format is an optimized data structure to store a collection of pre-commits and their signatures to be submitted as part of a commit message. Instead of storing an array of justifications, it uses the following format:
$$ {{J}{{{v}{{{0},\ldots{n}}}}}^{{{r},\text{comp}}}}:={\left({\left\lbrace{{V}{{{v}{{0}}}}^{{{r},{p}{c}}}},\ldots{{V}{{{v}{{n}}}}^{{{r},{p}{c}}}}\right\rbrace},{\left\lbrace{\left({\text{Sig}{{{v}{{0}}}}^{{{r},{p}{c}}}},{v}{{\text{id}{{0}}}}\right)},\ldots{\left({\text{Sig}{{{v}{{n}}}}^{{{r},{p}{c}}}},{v}{{\text{id}{{n}}}}\right)}\right\rbrace}\right)} $$
where
-
${{V}{{{v}{{i}}}}^{{{r},{p}{c}}}}$ is a 256-bit byte array containing the pre-commit vote of authority
${v}_{{i}}$ (Definition -def-num-ref-). -
${\text{Sig}{{{v}{{i}}}}^{{{r},{p}{c}}}}$ is a 512-bit byte array containing the pre-commit signature of authority
${v}_{{i}}$ (Definition -def-num-ref-). -
${v}{{\text{id}{{n}}}}$ is a 256-bit byte array containing the public key of authority
${v}_{{i}}$ .
:::
:::definition
A GRANDPA commit message for block
$$ {{M}{{v}}^{{{r},\text{Fin}}}}{\left({B}\right)}:=\text{Enc}{{\text{SC}}}{\left({r},\text{id}{{{\mathbb{{V}}}}},{{V}{{v}}^{{r}}}{\left({B}\right)},{{J}{{{v}{{{0},\ldots{n}}}}}^{{{r},\text{comp}}}}\right)} $$
where
-
${r}$ is an unsigned 64-bit integer indicating the round number (Definition -def-num-ref-). -
${id}_{{{\mathbb{{V}}}}}$ is the authority set Id (Definition -def-num-ref-). -
${{V}_{{v}}^{{r}}}{\left({B}\right)}$ is a 256-bit array containing the GRANDPA vote for block${B}$ (Definition -def-num-ref-). -
${{J}{{{v}{{{0},\ldots{n}}}}}^{{{r},\text{comp}}}}$ is the compacted GRANDPA justification containing observed pre-commit of authorities ${v}{{0}}$ to ${v}{{n}}$ (Definition -def-num-ref-).
This message is the sub-component of the GRANDPA gossip message (Definition -def-num-ref-) of type Id 1.
:::
Neighbor messages are sent to all connected peers but they are not repropagated on reception. A message should be sent whenever the values of the message change and at least every 5 minutes. The sender should take the recipient's state into account and avoid sending messages to peers that are using different voter sets or are in a different round. Messages received from a future voter set or round can be dropped and ignored.
:::definition
A GRANDPA Neighbor Message is defined as:
$$ {M}^{{\text{neigh}}}:=\text{Enc}{{\text{SC}}}{\left({v},{r},\text{id}{{{\mathbb{{V}}}}},{H}{{i}}{\left({B}{{\text{last}}}\right)}\right)} $$
where
-
${v}$ is an unsigned 8-bit integer indicating the version of the neighbor message, currently 1. -
${r}$ is an unsigned 64-bit integer indicating the round number (Definition -def-num-ref-). -
$\text{id}_{{{\mathbb{{V}}}}}$ is an unsigned 64-bit integer indicating the authority Id (Definition -def-num-ref-). -
${H}{{i}}{\left({B}{{\text{last}}}\right)}$ is an unsigned 32-bit integer indicating the block number of the last finalized block
${B}_{{\text{last}}}$ .
This message is the sub-component of the GRANDPA gossip message (Definition -def-num-ref-) of type Id 2.
:::
Whenever a Polkadot node detects that it is lagging behind the finality procedure, it needs to initiate a catch-up procedure. GRANDPA Neighbor messages (Definition -def-num-ref-) reveal the round number for the last finalized GRANDPA round which the node’s peers have observed. This provides the means to identify a discrepancy in the latest finalized round number observed among the peers. If such a discrepancy is observed, the node needs to initiate the catch-up procedure explained in Section -sec-num-ref-).
In particular, this procedure involves sending a catch-up request and processing catch-up response messages.
:::definition
A GRANDPA catch-up request message for round
$$ {{M}{{{i},{v}}}^{{{r},\text{Cat}-{q}}}}:=\text{Enc}{{\text{SC}}}{\left({r},\text{id}_{{{\mathbb{{V}}}}}\right)} $$
This message is the sub-component of the GRANDPA Gossip message (Definition -def-num-ref-) of type Id 3.
:::
:::definition
A GRANDPA catch-up response message for round
$$ {{M}{{{v},{i}}}^{{\text{Cat}-{s}}}}:=\text{Enc}{{\text{SC}}}{\left(\text{id}{{{\mathbb{{V}}}}},{r},{{J}{{{0},\ldots{n}}}^{{{r},\text{pv}}}}{\left({B}\right)},{{J}{{{0},\ldots{m}}}^{{{r},\text{pc}}}}{\left({B}\right)},{H}{{h}}{\left({B}'\right)},{H}_{{i}}{\left({B}'\right)}\right)} $$
Where
This message is the sub-component of the GRANDPA Gossip message (Definition -def-num-ref-) of type Id 4.
:::
This section defines the messages required for the BEEFY protocol (Section -sec-num-ref-). Those messages are sent over the /91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3/beefy/1 substream.
:::definition
A commitment,
-
${R}_{{h}}$ is the MMR root of all the block header hashes leading up to the latest, finalized block. - ${H}{{i}}{\left({B}{{\text{last}}}\right)}$ is the block number this commitment is for. Namely the latest, finalized block.
-
$\text{id}_{{{\mathbb{{V}}}}}$ is the current authority set Id (Definition -def-num-ref-). :::
:::definition A vote message, ${M}{{v}}$, is direct vote created by the Polkadot Host on every BEEFY round and is gossiped to its peers. The message is a datastructure of the following format: $$ {M}{{v}}=\text{Enc}{{\text{SC}}}{\left({C},{{A}{{\text{id}}}^{{\text{bfy}}}},{A}_{{\text{sig}}}\right)} $$ where
-
${C}$ is the BEEFY commitment (Definition -def-num-ref-). -
${{A}_{{\text{id}}}^{{\text{bfy}}}}$ is the ECDSA public key of the Polkadot Host. - ${A}{{\text{sig}}}$ is the signature created with ${{A}{{\text{id}}}^{{\text{bfy}}}}$ by signing the payload
${R}_{{h}}$ in${C}$ . :::
:::definition A signed commitment, ${M}{{\text{sc}}}$, is a datastructure of the following format: $$ {M}{{\text{SC}}}=\text{Enc}{{\text{SC}}}{\left({C},{S}{{n}}\right)} $$ $$ {S}{{n}}={\left({{A}{{0}}^{{\text{sig}}}},\ldots{{A}_{{n}}^{{\text{sig}}}}\right)} $$ where
-
${C}$ is the BEEFY commitment (Definition -def-num-ref-). - ${S}{{n}}$ is an array where its exact size matches the number of validators in the current authority set as specified by $\text{id}{{{\mathbb{{V}}}}}$ (Definition -def-num-ref-) in
${C}$ . Individual items are of the type Option (Definition -def-num-ref-), which can contain a signature of a validator which signed the same payload (${R}{{h}}$ in ${C}$) and is active in the current authority set. It’s critical that the signatures are sorted based on their corresponding public key entry in the authority set. For example, the signature of the validator at index 3 in the authority set must be placed at index 3 in ${S}{{n}}$. If not signature is available for that validator, then the Option variant is None inserted (Definition -def-num-ref-). This sorting allows clients to map public keys to their corresponding signatures. :::
:::definition A signed commitment witness, ${{M}{{\text{SC}}}^{{w}}}$, is a light version of the signed BEEFY commitment (Definition -def-num-ref-). Instead of containing the entire list of signatures, it only claims which validator signed the payload. The message is a datastructure of the following format: $$ {{M}{{\text{SC}}}^{{w}}}=\text{Enc}{{\text{SC}}}{\left({C},{V}{{{0},\ldots{n}}},{R}_{{\text{sig}}}\right)} $$ where
-
${C}$ is the BEEFY commitment (Definition -def-num-ref-). - ${V}{{{0},\ldots{n}}}$ is an array where its exact size matches the number of validators in the current authority set as specified by $\text{id}{{{\mathbb{{V}}}}}$ in
${C}$ . Individual items are booleans which indicate whether the validator has signed the payload (true) or not (false). It’s critical that the boolean indicators are sorted based on their corresponding public key entry in the authority set. For example, the boolean indicator of the validator at index 3 in the authority set must be placed at index 3 in${V}_{{n}}$ . This sorting allows clients to map public keys to their corresponding boolean indicators. -
${R}_{{\text{sig}}}$ is the MMR root of the signatures in the original signed BEEFY commitment (Definition -def-num-ref-).