-
Notifications
You must be signed in to change notification settings - Fork 220
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inbound DHT middleware #790
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR adds the vaious inbound DHT middleware components. The module is currently not part of the module tree as there are a number of changes to the comms and tari_middleware crates that need to go along with it. The `tari_comms_dht` crate adds DHT functionality to `tari_comms`. It provides two sets of middleware (_inbound_ and _outbound_) which process incoming requests and outgoing messages respectively. _Attaching to comms_ In `tari_comms`, incoming and outgoing messages are connected using two mpsc sender/receiver pairs. One for incoming messages (receiving `InboundMessage`s) and one for outbound messages (sending `OutboundMessage`s). The DHT module consists of two middleware layers (as in `tower_layer::Layer`) which form an inbound and outbound pipeline for augmenting messages. _Inbound Message Flow_ `InboundMessage`s are received from the incoming comms channel (as in the receiver side of of the mpsc channel which goes into `CommsBuilder::new().incoming_message_sink(sender)`). Typically, a `ServicePipeline` from the `tari_comms_middleware` crate is used to connect a stream from comms to the middleware service. `InboundMessage`(comms) -> _DHT Inbound Middleware_ -> `DhtInboundMessage`(domain) The DHT inbound middleware consist of: * `DeserializeMiddleware` deserializes the body of an `InboundMessage` into a `DhtEnvelope`. * `DecryptionMiddleware` attempts to decrypt the body of a `DhtEnvelope` if required. The result of that decryption (success or failure) is passed to the next service. * `ForwardMiddleware` uses the result of the decryption to determine if the message is destined for this node or not. If not, the message will be forwarded to the applicable peers using the OutboundRequester (i.e. the outbound DHT middleware). * `DhtHandlerMiddleware` handles DHT messages, such as `Join` and `Discover`. If the messages are _not_ DHT messages the `next_service` is called.
CjS77
approved these changes
Oct 1, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
CjS77
added a commit
that referenced
this pull request
Oct 1, 2019
Merge pull request #790 Inbound DHT middleware This PR adds the vaious inbound DHT middleware components. The module is currently not part of the module tree as there are a number of changes to the comms and tari_middleware crates that need to go along with it. Overview of middleware architecture The tari_comms_dht crate adds DHT functionality to tari_comms. It provides two sets of middleware (inbound and outbound) which process incoming requests and outgoing messages respectively. Attaching to comms In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs. One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages). The DHT module consists of two middleware layers (as in tower_layer::Layer) which form an inbound and outbound pipeline for augmenting messages. Inbound Message Flow InboundMessages are received from the incoming comms channel (as in the receiver side of of the mpsc channel which goes into CommsBuilder::new().incoming_message_sink(sender)). Typically, a ServicePipeline from the tari_comms_middleware crate is used to connect a stream from comms to the middleware service. InboundMessage(comms) -> DHT Inbound Middleware -> DhtInboundMessage(domain) The DHT inbound middleware consist of: DeserializeMiddleware deserializes the body of an InboundMessage into a DhtEnvelope. DecryptionMiddleware attempts to decrypt the body of a DhtEnvelope if required. The result of that decryption (success or failure) is passed to the next service. ForwardMiddleware uses the result of the decryption to determine if the message is destined for this node or not. If not, the message will be forwarded to the applicable peers using the OutboundRequester (i.e. the outbound DHT middleware). DhtHandlerMiddleware handles DHT messages, such as Join and Discover. If the messages are not DHT messages the next_service is called.
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. This PR contains the miriad changes and improvements relating to using middleware for inbound and outbound channels, completed implementation of DHT middleware making use of PRs #790 and #791. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
11 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the vaious inbound DHT middleware components.
The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.
Overview of middleware architecture
The
tari_comms_dht
crate adds DHT functionality totari_comms
.It provides two sets of middleware (inbound and outbound) which
process incoming requests and outgoing messages respectively.
Attaching to comms
In
tari_comms
, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.One for incoming messages (receiving
InboundMessage
s) and one for outbound messages (sendingOutboundMessage
s).The DHT module consists of two middleware layers (as in
tower_layer::Layer
) which forman inbound and outbound pipeline for augmenting messages.
Inbound Message Flow
InboundMessage
s are received from the incoming comms channel (as in the receiver side ofof the mpsc channel which goes into
CommsBuilder::new().incoming_message_sink(sender)
).Typically, a
ServicePipeline
from thetari_comms_middleware
crate is used to connecta stream from comms to the middleware service.
InboundMessage
(comms) -> DHT Inbound Middleware ->DhtInboundMessage
(domain)The DHT inbound middleware consist of:
DeserializeMiddleware
deserializes the body of anInboundMessage
into aDhtEnvelope
.DecryptionMiddleware
attempts to decrypt the body of aDhtEnvelope
if required. The result of that decryption(success or failure) is passed to the next service.
ForwardMiddleware
uses the result of the decryption to determine if the message is destined for this node ornot. If not, the message will be forwarded to the applicable peers using the OutboundRequester (i.e. the outbound
DHT middleware).
DhtHandlerMiddleware
handles DHT messages, such asJoin
andDiscover
. If the messages are not DHT messagesthe
next_service
is called.Motivation and Context
Ref #621
Ref #675
How Has This Been Tested?
Middlewares have unit tests although they will only be connected in subsequent PRs
Types of changes
Checklist:
development
branchcargo-fmt --all
before pushing