Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inbound DHT middleware #790

Merged
merged 1 commit into from
Oct 1, 2019
Merged

Inbound DHT middleware #790

merged 1 commit into from
Oct 1, 2019

Conversation

sdbondi
Copy link
Member

@sdbondi sdbondi commented Oct 1, 2019

Description

This PR adds the vaious inbound DHT middleware components.
The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.

Overview of middleware architecture

The tari_comms_dht crate adds DHT functionality to tari_comms.
It provides two sets of middleware (inbound and outbound) which
process incoming requests and outgoing messages respectively.

Attaching to comms

In tari_comms, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.
One for incoming messages (receiving InboundMessages) and one for outbound messages (sending OutboundMessages).

The DHT module consists of two middleware layers (as in tower_layer::Layer) which form
an inbound and outbound pipeline for augmenting messages.

Inbound Message Flow

InboundMessages are received from the incoming comms channel (as in the receiver side of
of the mpsc channel which goes into CommsBuilder::new().incoming_message_sink(sender)).
Typically, a ServicePipeline from the tari_comms_middleware crate is used to connect
a stream from comms to the middleware service.

InboundMessage(comms) -> DHT Inbound Middleware -> DhtInboundMessage(domain)

The DHT inbound middleware consist of:

  • DeserializeMiddleware deserializes the body of an InboundMessage into a DhtEnvelope.
  • DecryptionMiddleware attempts to decrypt the body of a DhtEnvelope if required. The result of that decryption
    (success or failure) is passed to the next service.
  • ForwardMiddleware uses the result of the decryption to determine if the message is destined for this node or
    not. If not, the message will be forwarded to the applicable peers using the OutboundRequester (i.e. the outbound
    DHT middleware).
  • DhtHandlerMiddleware handles DHT messages, such as Join and Discover. If the messages are not DHT messages
    the next_service is called.

Motivation and Context

Ref #621
Ref #675

How Has This Been Tested?

Middlewares have unit tests although they will only be connected in subsequent PRs

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Feature refactor (No new feature or functional changes, but performance or technical debt improvements)
  • New Tests
  • Documentation

Checklist:

  • I'm merging against the development branch
  • I ran cargo-fmt --all before pushing
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • [~] I have added tests to cover my changes.

This PR adds the vaious inbound DHT middleware components.
The module is currently not part of the module tree as
there are a number of changes to the comms and tari_middleware
crates that need to go along with it.

The `tari_comms_dht` crate adds DHT functionality to `tari_comms`.
It provides two sets of middleware (_inbound_ and _outbound_) which
process incoming requests and outgoing messages respectively.

_Attaching to comms_

In `tari_comms`, incoming and outgoing messages are connected using two mpsc sender/receiver pairs.
One for incoming messages (receiving `InboundMessage`s) and one for outbound messages (sending `OutboundMessage`s).

The DHT module consists of two middleware layers (as in `tower_layer::Layer`) which form
an inbound and outbound pipeline for augmenting messages.

_Inbound Message Flow_

`InboundMessage`s are received from the incoming comms channel (as in the receiver side of
of the mpsc channel which goes into `CommsBuilder::new().incoming_message_sink(sender)`).
Typically, a `ServicePipeline` from the `tari_comms_middleware` crate is used to connect
a stream from comms to the middleware service.

`InboundMessage`(comms) -> _DHT Inbound Middleware_ -> `DhtInboundMessage`(domain)

The DHT inbound middleware consist of:
* `DeserializeMiddleware` deserializes the body of an `InboundMessage` into a `DhtEnvelope`.
* `DecryptionMiddleware` attempts to decrypt the body of a `DhtEnvelope` if required. The result of that decryption
(success or failure) is passed to the next service.
* `ForwardMiddleware` uses the result of the decryption to determine if the message is destined for this node or
not. If not, the message will be forwarded to the applicable peers using the OutboundRequester (i.e. the outbound
DHT middleware).
* `DhtHandlerMiddleware` handles DHT messages, such as `Join` and `Discover`. If the messages are _not_ DHT messages
the `next_service` is called.
Copy link
Collaborator

@CjS77 CjS77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

CjS77 added a commit that referenced this pull request Oct 1, 2019
Merge pull request #790

Inbound DHT middleware

This PR adds the vaious inbound DHT middleware components.  The module is
currently not part of the module tree as there are a number of changes to the
comms and tari_middleware crates that need to go along with it.

Overview of middleware architecture

The tari_comms_dht crate adds DHT functionality to tari_comms.  It provides two
sets of middleware (inbound and outbound) which process incoming requests and
outgoing messages respectively.

Attaching to comms

In tari_comms, incoming and outgoing messages are connected using two mpsc
sender/receiver pairs.  One for incoming messages (receiving InboundMessages)
and one for outbound messages (sending OutboundMessages).

The DHT module consists of two middleware layers (as in tower_layer::Layer)
which form an inbound and outbound pipeline for augmenting messages.

Inbound Message Flow

InboundMessages are received from the incoming comms channel (as in the
receiver side of of the mpsc channel which goes into
CommsBuilder::new().incoming_message_sink(sender)).  Typically, a
ServicePipeline from the tari_comms_middleware crate is used to connect a
stream from comms to the middleware service.

InboundMessage(comms) -> DHT Inbound Middleware -> DhtInboundMessage(domain)

The DHT inbound middleware consist of:

DeserializeMiddleware deserializes the body of an InboundMessage into a
DhtEnvelope.  DecryptionMiddleware attempts to decrypt the body of a
DhtEnvelope if required. The result of that decryption (success or failure) is
passed to the next service.  ForwardMiddleware uses the result of the
decryption to determine if the message is destined for this node or not. If
not, the message will be forwarded to the applicable peers using the
OutboundRequester (i.e. the outbound DHT middleware).  DhtHandlerMiddleware
handles DHT messages, such as Join and Discover. If the messages are not DHT
messages the next_service is called.
@CjS77 CjS77 merged commit db7da5b into development Oct 1, 2019
@CjS77 CjS77 deleted the sb-comms-dht-I branch October 1, 2019 13:04
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
Unfortunately a lot had to change in order to implement the
middleware architecture, this PR contains those changes.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
sdbondi added a commit that referenced this pull request Oct 1, 2019
This PR uses the middlewares in #790 and #791.
This PR contains the miriad changes and improvements
relating to using middleware for inbound and outbound
channels, completed implementation of DHT middleware
making use of PRs #790 and #791.

- Big refactor of messages being passed around
 (comms-level and dht-level messages)
- OMS is responsible for signing comms-level header for peers
- DHT is responsible for signing dht-level header
- Pulled out middleware from comms (replaced with inbound and outbound
  mpsc channel)
- Moved domain-level middleware into tari_p2p
- Moved DHT-level middleware from tari_middleware to tari_middleware_dht
- Bug fix for OMS exponential backoff
- Setup DHT module in tari_p2p initialize_comms
- Moved broadcast strategy into dht
- Updated pingpong example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants