Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outbound DHT middleware #791

Merged
merged 1 commit into from
Oct 1, 2019
Merged

Outbound DHT middleware #791

merged 1 commit into from
Oct 1, 2019

Conversation

sdbondi
Copy link
Member

@sdbondi sdbondi commented Oct 1, 2019

Description

This PR adds the vaious outbound DHT middleware components.

The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.

Overview of middleware architecture

The tari_comms_dht crate adds DHT functionality to tari_comms.
It provides two sets of middleware (inbound and outbound) which
process incoming requests and outgoing messages respectively.

Attaching to comms

In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.
One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages).

The DHT module consists of two middleware layers (as in tower_layer::Layer) which form
an inbound and outbound pipeline for augmenting messages.

Outbound Message Flow

OutboundMessages are sent to the outgoing comms channel (as in the receiver side of
of the mpsc channel which goes into CommsBuilder::new().outgoing_message_stream(receiver)).
Typically, a ServicePipeline from the tari_comms_middleware crate is used to connect
a stream from the domain-level to the middleware service and a SinkMiddleware to connect
the middleware to the OMS in comms. Outbound requests to the DHT middleware are furnished by
the OutboundMessageRequester, obtained from the Dht::outbound_requester factory method.

DhtOutboundRequest (domain) -> DHT Outbound Middleware -> OutboundMessage (comms)

The DHT outbound middleware consist of:

  • BroadcastMiddleware produces multiple outbound messages according on the BroadcastStrategy from the received
    DhtOutboundRequest message. The next_service is called for each resulting message.
  • EncryptionMiddleware encrypts the body of a message if DhtMessagheFlags::ENCRYPTED is given. The result is
    passed onto the next_service.
  • SerializeMiddleware wraps the body in a DhtEnvelope, serializes the result, constructs an OutboundMessage
    and calls next_service. Typically, next_service will be a SinkMiddleware which send the message to the comms
    OMS.

Motivation and Context

Ref #621
Ref #675

How Has This Been Tested?

Middlewares have unit tests although they will only be connected in subsequent PRs

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Feature refactor (No new feature or functional changes, but performance or technical debt improvements)
  • New Tests
  • Documentation

Checklist:

  • I'm merging against the development branch
  • I ran cargo-fmt --all before pushing
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

This PR adds the vaious outbound DHT middleware components.

The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.

Overview of middleware architecture

The tari_comms_dht crate adds DHT functionality to tari_comms.
It provides two sets of middleware (inbound and outbound) which
process incoming requests and outgoing messages respectively.

Attaching to comms

In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.
One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages).

The DHT module consists of two middleware layers (as in tower_layer::Layer) which form
an inbound and outbound pipeline for augmenting messages.

 Outbound Message Flow

`OutboundMessage`s are sent to the outgoing comms channel (as in the receiver side of
of the mpsc channel which goes into `CommsBuilder::new().outgoing_message_stream(receiver)`).
Typically, a `ServicePipeline` from the `tari_comms_middleware` crate is used to connect
a stream from the domain-level to the middleware service and a `SinkMiddleware` to connect
the middleware to the OMS in comms. Outbound requests to the DHT middleware are furnished by
the `OutboundMessageRequester`, obtained from the `Dht::outbound_requester` factory method.

`DhtOutboundRequest` (domain) -> _DHT Outbound Middleware_ -> `OutboundMessage` (comms)

The DHT outbound middleware consist of:
* `BroadcastMiddleware` produces multiple outbound messages according on the `BroadcastStrategy` from the received
`DhtOutboundRequest` message. The `next_service` is called for each resulting message.
* `EncryptionMiddleware` encrypts the body of a message if `DhtMessagheFlags::ENCRYPTED` is given. The result is
passed onto the `next_service`.
* `SerializeMiddleware` wraps the body in a `DhtEnvelope`, serializes the result, constructs an `OutboundMessage`
and calls `next_service`. Typically, `next_service` will be a `SinkMiddleware` which send the message to the comms
OMS.
Copy link
Collaborator

@CjS77 CjS77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

CjS77 added a commit that referenced this pull request Oct 1, 2019
This PR adds the vaious outbound DHT middleware components.

The module is currently not part of the module tree as there are a number of
changes to the comms and tari_middleware crates that need to go along with it.

Overview of middleware architecture

The tari_comms_dht crate adds DHT functionality to tari_comms.  It provides two
sets of middleware (inbound and outbound) which process incoming requests and
outgoing messages respectively.

Attaching to comms

In tari_comms, incoming and outgoing messages are connected using two mpsc
sender/receiver pairs.  One for incoming messages (receiving InboundMessages)
and one for outbound messages (sending OutboundMessages).

The DHT module consists of two middleware layers (as in tower_layer::Layer)
which form an inbound and outbound pipeline for augmenting messages.

Outbound Message Flow

OutboundMessages are sent to the outgoing comms channel (as in the receiver
side of of the mpsc channel which goes into
CommsBuilder::new().outgoing_message_stream(receiver)).  Typically, a
ServicePipeline from the tari_comms_middleware crate is used to connect a
stream from the domain-level to the middleware service and a SinkMiddleware to
connect the middleware to the OMS in comms. Outbound requests to the DHT
middleware are furnished by the OutboundMessageRequester, obtained from the
Dht::outbound_requester factory method.

DhtOutboundRequest (domain) -> DHT Outbound Middleware -> OutboundMessage
(comms)

The DHT outbound middleware consist of:

BroadcastMiddleware produces multiple outbound messages according on the
BroadcastStrategy from the received DhtOutboundRequest message. The
next_service is called for each resulting message.  EncryptionMiddleware
encrypts the body of a message if DhtMessagheFlags::ENCRYPTED is given. The
result is passed onto the next_service.  SerializeMiddleware wraps the body in
a DhtEnvelope, serializes the result, constructs an OutboundMessage and calls
next_service. Typically, next_service will be a SinkMiddleware which send the
message to the comms OMS.
@CjS77 CjS77 merged commit c50986b into development Oct 1, 2019
@CjS77 CjS77 deleted the sb-comms-dht-outbound branch October 1, 2019 13:06
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
This PR contains the miriad changes and improvements
relating to using middleware for inbound and outbound
channels, completed implementation of DHT middleware
making use of PRs #790 and #791.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants