-
Notifications
You must be signed in to change notification settings - Fork 220
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Outbound DHT middleware #791
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR adds the vaious outbound DHT middleware components. The module is currently not part of the module tree as there are a number of changes to the comms and tari_middleware crates that need to go along with it. Overview of middleware architecture The tari_comms_dht crate adds DHT functionality to tari_comms. It provides two sets of middleware (inbound and outbound) which process incoming requests and outgoing messages respectively. Attaching to comms In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs. One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages). The DHT module consists of two middleware layers (as in tower_layer::Layer) which form an inbound and outbound pipeline for augmenting messages. Outbound Message Flow `OutboundMessage`s are sent to the outgoing comms channel (as in the receiver side of of the mpsc channel which goes into `CommsBuilder::new().outgoing_message_stream(receiver)`). Typically, a `ServicePipeline` from the `tari_comms_middleware` crate is used to connect a stream from the domain-level to the middleware service and a `SinkMiddleware` to connect the middleware to the OMS in comms. Outbound requests to the DHT middleware are furnished by the `OutboundMessageRequester`, obtained from the `Dht::outbound_requester` factory method. `DhtOutboundRequest` (domain) -> _DHT Outbound Middleware_ -> `OutboundMessage` (comms) The DHT outbound middleware consist of: * `BroadcastMiddleware` produces multiple outbound messages according on the `BroadcastStrategy` from the received `DhtOutboundRequest` message. The `next_service` is called for each resulting message. * `EncryptionMiddleware` encrypts the body of a message if `DhtMessagheFlags::ENCRYPTED` is given. The result is passed onto the `next_service`. * `SerializeMiddleware` wraps the body in a `DhtEnvelope`, serializes the result, constructs an `OutboundMessage` and calls `next_service`. Typically, `next_service` will be a `SinkMiddleware` which send the message to the comms OMS.
CjS77
approved these changes
Oct 1, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
CjS77
added a commit
that referenced
this pull request
Oct 1, 2019
This PR adds the vaious outbound DHT middleware components. The module is currently not part of the module tree as there are a number of changes to the comms and tari_middleware crates that need to go along with it. Overview of middleware architecture The tari_comms_dht crate adds DHT functionality to tari_comms. It provides two sets of middleware (inbound and outbound) which process incoming requests and outgoing messages respectively. Attaching to comms In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs. One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages). The DHT module consists of two middleware layers (as in tower_layer::Layer) which form an inbound and outbound pipeline for augmenting messages. Outbound Message Flow OutboundMessages are sent to the outgoing comms channel (as in the receiver side of of the mpsc channel which goes into CommsBuilder::new().outgoing_message_stream(receiver)). Typically, a ServicePipeline from the tari_comms_middleware crate is used to connect a stream from the domain-level to the middleware service and a SinkMiddleware to connect the middleware to the OMS in comms. Outbound requests to the DHT middleware are furnished by the OutboundMessageRequester, obtained from the Dht::outbound_requester factory method. DhtOutboundRequest (domain) -> DHT Outbound Middleware -> OutboundMessage (comms) The DHT outbound middleware consist of: BroadcastMiddleware produces multiple outbound messages according on the BroadcastStrategy from the received DhtOutboundRequest message. The next_service is called for each resulting message. EncryptionMiddleware encrypts the body of a message if DhtMessagheFlags::ENCRYPTED is given. The result is passed onto the next_service. SerializeMiddleware wraps the body in a DhtEnvelope, serializes the result, constructs an OutboundMessage and calls next_service. Typically, next_service will be a SinkMiddleware which send the message to the comms OMS.
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. Unfortunately a lot had to change in order to implement the middleware architecture, this PR contains those changes. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
sdbondi
added a commit
that referenced
this pull request
Oct 1, 2019
This PR uses the middlewares in #790 and #791. This PR contains the miriad changes and improvements relating to using middleware for inbound and outbound channels, completed implementation of DHT middleware making use of PRs #790 and #791. - Big refactor of messages being passed around (comms-level and dht-level messages) - OMS is responsible for signing comms-level header for peers - DHT is responsible for signing dht-level header - Pulled out middleware from comms (replaced with inbound and outbound mpsc channel) - Moved domain-level middleware into tari_p2p - Moved DHT-level middleware from tari_middleware to tari_middleware_dht - Bug fix for OMS exponential backoff - Setup DHT module in tari_p2p initialize_comms - Moved broadcast strategy into dht - Updated pingpong example
11 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the vaious outbound DHT middleware components.
The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.
Overview of middleware architecture
The tari_comms_dht crate adds DHT functionality to tari_comms.
It provides two sets of middleware (inbound and outbound) which
process incoming requests and outgoing messages respectively.
Attaching to comms
In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.
One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages).
The DHT module consists of two middleware layers (as in tower_layer::Layer) which form
an inbound and outbound pipeline for augmenting messages.
Outbound Message Flow
OutboundMessage
s are sent to the outgoing comms channel (as in the receiver side ofof the mpsc channel which goes into
CommsBuilder::new().outgoing_message_stream(receiver)
).Typically, a
ServicePipeline
from thetari_comms_middleware
crate is used to connecta stream from the domain-level to the middleware service and a
SinkMiddleware
to connectthe middleware to the OMS in comms. Outbound requests to the DHT middleware are furnished by
the
OutboundMessageRequester
, obtained from theDht::outbound_requester
factory method.DhtOutboundRequest
(domain) -> DHT Outbound Middleware ->OutboundMessage
(comms)The DHT outbound middleware consist of:
BroadcastMiddleware
produces multiple outbound messages according on theBroadcastStrategy
from the receivedDhtOutboundRequest
message. Thenext_service
is called for each resulting message.EncryptionMiddleware
encrypts the body of a message ifDhtMessagheFlags::ENCRYPTED
is given. The result ispassed onto the
next_service
.SerializeMiddleware
wraps the body in aDhtEnvelope
, serializes the result, constructs anOutboundMessage
and calls
next_service
. Typically,next_service
will be aSinkMiddleware
which send the message to the commsOMS.
Motivation and Context
Ref #621
Ref #675
How Has This Been Tested?
Middlewares have unit tests although they will only be connected in subsequent PRs
Types of changes
Checklist:
development
branchcargo-fmt --all
before pushing