Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Networking roadmap #1692

Closed
5 of 12 tasks
tomaka opened this issue Feb 5, 2019 · 4 comments
Closed
5 of 12 tasks

Networking roadmap #1692

tomaka opened this issue Feb 5, 2019 · 4 comments
Labels
J1-meta A specific issue for grouping tasks or bugs of a specific category.
Milestone

Comments

@tomaka
Copy link
Contributor

tomaka commented Feb 5, 2019

Here are some steps that I have in mind for the network crate, ordered in order to have a smooth transition. Feel welcome to comment.

  • Make Service and ServiceEvent generic over the message type, and send/receive messages instead of Bytes objects.
  • Add a trait in network-libp2p that is implemented on Message and allows retrieving the RequestId.
  • Make network-libp2p/src/custom_proto/handler.rs open a new substream for each request. Whether to wait for an answer and how to associate substreams with requests is done using the trait mentioned in the previous point. (Rework network/src/protocol.rs to use libp2p's multiplexing #1517)
  • Remove the Clogged event.
  • (blurry) Rewrite network/src/protocol.rs as a libp2p NetworkBehaviour. Instead of passing a RegisteredProtocol to network-libp2p, one can pass a NetworkBehaviour. Move network-libp2p/src/custom_protocol to network. The messages generated by the API of network-libp2p's Service would be the events generated by the network behaviour. We would also require a trait in addition to NetworkBehaviour so that we can register discovered nodes to the behaviour.
  • Move some logic from the protocol.rs NetworkBehaviour into the ProtocolsHandler, such as the Status message. After this point, Have an upgrade strategy for the network protocol #1518 and Add a networking message to enable/disable the Substrate protocol on connections #1702 can be tackled.
  • Extract the gossiping from protocol.rs to use a different protocol from the rest (the rest meaning things such as block or header requests).
  • (blurry) Use libp2p's floodsub, episub or gossipsub instead of consensus_gossip (More sophisticated gossip for consensus messages #366).
  • Change everything to use futures and messages passing instead of Arcs (Rewrite sync and network layers to use futures #834), and use tokio-timer instead of maintain_peers, tick or garbage_collect.
  • Merge network and network-libp2p?

Later:

@tomaka
Copy link
Contributor Author

tomaka commented Mar 25, 2019

The current step is more or less "Rewrite network/src/protocol.rs as a libp2p NetworkBehaviour" now.

For this step, I'm going to change a bit the API of substrate-network-libp2p.
At the moment, it is an opaque state machine where you open/close connections with remote, and read/write messages.

After this step, the ServiceTask struct would accept two template parameters: a discovery protocol, and a gossiping protocol. They would both be defined by traits in substrate-network-libp2p. The discovery protocol would generate events that are then indirectly reflected on the gossiping protocol, and the gossiping protocol would generate events that are propagated to the outside of the ServiceTask.

The text mentions "Move network-libp2p/src/custom_protocol to network.", but I wouldn't do that in the end. The substrate-network-libp2p crate would provide implementations of the discovery and gossiping traits that could be used by whatever code uses substrate-network-libp2p.

This defines a bit more precisely the role of substrate-network-libp2p: provide a ServiceTask that combines discovery and gossiping, plus all the various ways of doing discovery and gossiping that we would support.

The next steps would then to move more content from protocol.rs to substrate-network-libp2p.

@gterzian
Copy link
Contributor

@tomaka Do you need help with "Rewrite network/src/protocol.rs as a libp2p NetworkBehaviour"?

@tomaka
Copy link
Contributor Author

tomaka commented Apr 15, 2019

The first step was libp2p/rust-libp2p#1046 in libp2p. Once a libp2p version is published and once most of the networking issues are fixed, I'll finish my work in progress PR.

@tomaka
Copy link
Contributor Author

tomaka commented Apr 25, 2019

I'm going to close this issue, as half of it is implemented and only #2374 is still important.

@tomaka tomaka closed this as completed Apr 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
J1-meta A specific issue for grouping tasks or bugs of a specific category.
Projects
None yet
Development

No branches or pull requests

3 participants