forked from informalsystems/hermes
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ADR: Domain Decomposition (informalsystems#162)
* Initial sketch of domain decomposition * Add dependencies outline * Add potential components names next to operations * update based on relayer spec * Align with relayer-spike * update the status * fix typos and add the second foreign client * Reviewed ADR 004 Co-authored-by: Romain Ruetschi <romain@informal.systems> Co-authored-by: Anca Zamfir <zamfiranca@gmail.com> Co-authored-by: Adi Seredinschi <adi@informal.systems>
- Loading branch information
1 parent
e57f948
commit 73b54f0
Showing
2 changed files
with
309 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
305 changes: 305 additions & 0 deletions
305
docs/architecture/adr-004-relayer-domain-decomposition.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,305 @@ | ||
# ADR 004: Relayer Domain Decomposition | ||
|
||
## Changelog | ||
* 21.7.2020: Initial sketch | ||
* 27.7.2020: Dependencies outline | ||
* 5.10.2020: Update based on sketch | ||
* 2.11.2020: Reviewed & accepted | ||
|
||
## Context | ||
|
||
The IBC handlers queries and relayer are defined loosely in the [specs]. | ||
The goal of this ADR is to provide clarity around the basic domain objects | ||
from the perspective of the relayer, | ||
their interfaces as well as dependencies between them in order to | ||
guide the implementation. The success criteria for the decomposition is | ||
how well it can be tested. It's expected that any decomposition will | ||
lend itself to tight unit tests allowing more collaborators make change | ||
in the code base with confidence. | ||
|
||
## Decision | ||
The decomposition should be motivated by what we want to test and what | ||
we need to mock out to exercise the core logic. | ||
|
||
We want to be able to test the following high-level functions: | ||
|
||
* Client create and update | ||
* With different chain states | ||
* Connection handshake | ||
* With different chain states | ||
* Channel Setup | ||
* With different chain states | ||
* Datagram Construction | ||
* With different chain states | ||
* Datagram to Transaction | ||
* Batching | ||
* Signing | ||
* Datagram Submission | ||
* With different chain states | ||
* Missing Client Updates | ||
* With Missing Proofs | ||
* Handlers (datagrams, chain state) -> events | ||
* Handling the batch of datagrams | ||
* With different chain states | ||
* Specifically, the key value store | ||
* Produce events | ||
|
||
## Dependencies | ||
|
||
In this section, we map the operations which need to be performed at different | ||
stages of both the relayer and the IBC handlers. This gives an outline | ||
of what low-level operations and dependencies need to be mocked out to test each | ||
stage in isolation, and will inform the design of the various traits needed | ||
to mock those out. | ||
|
||
Not all stages are listed here because the operations and dependencies | ||
outlined below cover all the possible dependencies at each stage. | ||
|
||
### Initializing a connection from the relayer | ||
|
||
- Need a relayer configuration (relayer.toml) | ||
- Query chain B for its commitment prefix (ABCI query) | ||
- Send `MsgConnectionOpenInit` message to chain A (transaction) | ||
|
||
### `ConnOpenInit` (Handler) | ||
|
||
- Provable store | ||
- Private store | ||
|
||
### `updateIBCClient` (Relayer) | ||
|
||
- get the latest height from chain A (Query) | ||
- get client consensus state from chain B (Query) | ||
- get latest header + minimal set from chain A (Light Client) | ||
- verify client state proof (Prover) | ||
- create and submit datagrams to update B's view of A (Message Builder, Transaction) | ||
- replace full node for B with other full node (PeerList) | ||
- create and submit proof of fork (Fork Evidence Reporter) | ||
- wait for UpdateClient event (Event Subscription) | ||
|
||
### `pendingDatagrams` (Relayer) | ||
Builds the datagrams required by the given on-chain states. | ||
For connection datagrams: | ||
- get connection objects from chain A (Query) | ||
- get connection objects from chain B (Query) | ||
- get proof\* of connection state (e.g. `Init`) from chain A (Query, Prover, Light Client) | ||
- get proof\* of client state and consensus state from chain A (Query, Prover, Light Client) | ||
- \* involves querying the chain + get header/minimal set + verify proof | ||
- build the next message in the connection handshake, e.g. `ConnOpenTry` (Message Builder) | ||
|
||
Channel datagrams are built similarly. Packet datagrams are triggered by events, and they are detailed in the Link section below. | ||
|
||
### IBC Module | ||
|
||
For every a transaction in a block of height H: | ||
|
||
- call appropriate handler (this is realized by ICS26 routing sub-module), | ||
- If handler succeeds (transaction does not abort), then | ||
apply the updates to the key-value store (provable & private), and also | ||
get the current height H and emit appropriate events. | ||
|
||
## Objects | ||
|
||
The main domain objects in the relayer (`ForeignClient`, `Connection`, `Channel`) | ||
will be created as concrete types which contain their configuration. | ||
These objects are the relayer's representation of different parts of state from the two chains. | ||
Dependencies between types indicate runtime dependencies of the chain | ||
state. For instance, objects parameterized by a `ForeignClient` type, such as a `Connection`, | ||
have the pre-condition that the runtime completed client creation before operating with | ||
those objects. | ||
|
||
### ChainHandle | ||
|
||
The ChainHandle trait is a local representation of a chain state on the relayer. | ||
The API of a ChainHandle provides reliable access to chain state whether | ||
crashed, constructed or queried. We envision a mock version of a chain | ||
will be used to test both handler and relayer logic ([#158]). | ||
|
||
The method set of the ChainHandle trait will reflect specific needs and not | ||
intermediate representations. Query and light client verification | ||
concerns will be internal to the chain handle implementation and not exposed | ||
via this API. Users of a ChainHandle implementation (i.e., relayer methods) | ||
can assume verified data or receive an Error and act appropriately. | ||
|
||
```rust | ||
trait ChainHandle { | ||
// Generate a packet | ||
fn create_packet(&self, event: IBCEvent) -> Result<Packet, ChainError>; | ||
|
||
// Fetch the height of an IBC client hosted by a chain | ||
// - query the consensus_state of src on dst | ||
// - query the highest consensus_state | ||
// - verify if with the light client | ||
// - return the height | ||
fn get_height(&self, client: &ForeignClient) -> Result<Height, ChainError>; | ||
|
||
// Submit a transaction to a chains underlying full node | ||
fn submit(&self, transaction: EncodedTransaction) -> Result<(), ChainError>; | ||
|
||
// ... | ||
} | ||
|
||
``` | ||
|
||
### Connection | ||
|
||
```rust | ||
impl Connection { | ||
fn new( | ||
chain_a: &ChainHandle, | ||
chain_b: &ChainHandle, | ||
foreign_client_a: &ForeignClient, | ||
foreign_client_b: &ForeignClient, | ||
config: ConnectionConfig) | ||
-> Result<Connection, Error> { | ||
// Establish a connection between ChainA and ChainB via ICS 3 handshake. | ||
// For a first version this can be completely synchronous (a blocking call). | ||
} | ||
} | ||
``` | ||
|
||
### Channel | ||
```rust | ||
impl Channel { | ||
fn new( | ||
chain_a: &ChainHandle, | ||
chain_b: &ChainHandle, | ||
connection: &Connection, | ||
config: ChannelConfig) | ||
-> Result<Channel, Error> { | ||
// Establish a channel between two modules (i.e., ICS4 handshake). | ||
} | ||
} | ||
``` | ||
|
||
## Link | ||
|
||
A link is the object that connects two specific modules on separate chains. | ||
Links are responsible for relaying packets from `chain_a` | ||
to `chain_b` and are therefore uni-directional. A single relayer process | ||
should be able to support multiple link instances and each link should | ||
run in its own thread. Links depend on `ForeignClient`s, | ||
`Connection` and `Channel`. | ||
|
||
```rust | ||
struct Link { | ||
src_chain: &ChainHandle, | ||
dst_chain: &ChainHandle, | ||
channel: &Channel, | ||
} | ||
|
||
impl Link { | ||
fn new(channel: &Channel, config: LinkConfig) | ||
-> Link { | ||
// ... | ||
} | ||
|
||
/// Relay Link specific packets from src_chain to dst_chain | ||
/// Expect this to run in a thread | ||
fn run(self) -> Error { | ||
let subscription = self.src_chain.subscribe(&self.channel); | ||
|
||
for (target_height, events) in subscription.iter() { | ||
// ... | ||
|
||
let datagrams = events.map(|event| { | ||
Datagram::Packet(self.dsrc_chain.build_packet(target_height, event)) | ||
}); | ||
|
||
for attempt in self.config_submission_attempts { | ||
let current_height = self.dst_chain.get_height(&self.connection.channel.foreign_client)?; | ||
let signed_headers = self.src_chain.get_minimal_set(current_height, target_height)?; | ||
|
||
let mut attempt_datagrams = datagrams.clone(); | ||
attempt_datagrams.push(Datagram::ClientUpdat(ClientUpdate::new(signed_headers))); | ||
|
||
let transaction = Transaction::new(datagram); | ||
self.dst_chain.submit(transaction.sign().encode())?; | ||
} | ||
|
||
} | ||
} | ||
} | ||
``` | ||
|
||
### Example Main | ||
|
||
Example of the initializing of a single link between two chains. Each | ||
chain has its own runtime and exposes a `handle` to communicate with | ||
that runtime from different threads. There are dependencies between | ||
ForeignClients, Connections, Channels and Links which are encoded in the | ||
type system. The construction of them reflects that their corresponding | ||
handshake protocol has completed successfully. | ||
|
||
```rust | ||
fn main() -> Result<(), Box<dyn Error>> { | ||
let src_chain = ChainRuntime::new(); | ||
let dst_chain = ChainRuntime::new(); | ||
|
||
/// chains expose handlers for commuicating with the chain related runtime | ||
/// which move into their own threads | ||
let src_chain_handle = src_chain.handle(); | ||
thread::spawn(move || { | ||
src_chain.run().unwrap(); | ||
}); | ||
|
||
let dst_chain_handle = dst_chain.handle(); | ||
thread::spawn(move || { | ||
// What should we do on return here? | ||
dst_chain.run().unwrap(); | ||
}); | ||
|
||
let src_foreign_client_on_dst = ForeignClient::new( | ||
&src_chain_handle, | ||
&dst_chain_handle, | ||
ForeignClientConfig::default())?; | ||
|
||
let dst_foreign_client_on_src = ForeignClient::new( | ||
&src_chain_handle, | ||
&dst_chain_handle, | ||
ForeignClientConfig::default())?; | ||
|
||
let connection = Connection::new( | ||
&src_chain_handle, | ||
&dst_chain_handle, | ||
dst_foreign_client_on_src, | ||
src_foreign_client_on_dst, | ||
ConnectionConfig::default()).unwrap(); | ||
|
||
let channel = Channel::new( | ||
&src_chain_handle, | ||
&dst_chain_handle, | ||
connection, | ||
ChannelConfig::default()).unwrap(); | ||
|
||
let link = Link::new( | ||
src_chain_handle, | ||
dst_chain_handle, | ||
channel, | ||
LinkConfig::default())?; | ||
|
||
link.run()?; | ||
|
||
Ok(()) | ||
} | ||
``` | ||
|
||
## Status | ||
|
||
- Accepted (first implementation in [#335](https://github.com/informalsystems/ibc-rs/pull/335)). | ||
|
||
## Consequences | ||
|
||
### Positive | ||
* Clean abstractions an isolation from IO | ||
* Handshakes are correct by construction | ||
* Sane error handling | ||
|
||
### Negative | ||
|
||
### Neutral | ||
|
||
## References | ||
|
||
[specs]: https://github.com/cosmos/ics/tree/master/spec | ||
[#158]: https://github.com/informalsystems/ibc-rs/issues/158 |