Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onion format variation a-la @roasbeef (vs #593) #604

Closed
wants to merge 6 commits into from
Closed

Onion format variation a-la @roasbeef (vs #593) #604

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e526a82
bolt04: Formatting cleanup and fold clarifications into conventions
cdecker Mar 28, 2019
6cd50f3
bolt04: Introduce the notion of frames and make payloads variable
cdecker Mar 28, 2019
9807253
bolt04: Update the technical description of the multi-frame proposal
cdecker Mar 29, 2019
101f0e1
bolt04: Move the test vectors into JSON documents
cdecker Mar 29, 2019
89fb441
bolt04: Update spell check dictionary
cdecker Mar 29, 2019
3230264
BOLT 4: leave legacy realm byte alone, use values 1-15 for multi-fram…
rustyrussell May 6, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 67 additions & 21 deletions 04-onion-routing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,17 +181,14 @@ Each `hop_payload` has the following structure:

1. type: `hop_payload`
2. data:
* [`1`: `num_frames_and_realm`]
* [`1`: `realm`]
* [`raw_payload_len`: `raw_payload`]
* [`padding_len`: `padding`]
* [`32`: `HMAC`]

Notice that since the `hop_payload` is instantiated once per hop, the subscript `_i` may be used in the remainder of this document to refer to the `hop_payload` and its fields destined for hop `i`.

The `hop_payload` consists of at least one `frame` followed by up to 15 additional `frame`s.
The number of additional frames allocated to the current hop is determined by the 4 most significant bits of `num_frames_and_realm`, while the 4 least significant bits determine the payload format.
Therefore the number of frames allocated to the current hop is given by `num_frames = (num_frames_and_realm >> 4) + 1`.
For simplification we will use `hop_payload_len` to refer to `num_frames * FRAME_SIZE`.
The `hop_payload` consists of at least one `frame` followed by up to 15 additional `frame`s. For simplification we will use `hop_payload_len` to refer to `num_frames * FRAME_SIZE`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the entire byte is available, why restrict it to 15 instead of ~20 which lets nodes utilize all the available space?


In order to have sufficient space to serialized the `raw_payload` into the `hop_payload` while minimizing the number of used frames the number of frames used for a single `hop_payload` MUST be equal to

Expand All @@ -200,24 +197,37 @@ In order to have sufficient space to serialized the `raw_payload` into the `hop_
The payload format determines how the `raw_payload` should be interpreted (see below for currently defined formats), and how much padding is added.
In order to position the `HMAC` in the last 32 bytes of the `hop` the `raw_payload` MUST be followed by `padding_len = (num_frames * FRAME_SIZE - 1 - raw_payload_len - 32)` `0x00`-bytes.

The `realm` is specified as `num_frames_and_realm & 0x0F`.
It determines the format of the `raw_payload` field; the following `realm`s are currently defined.

| `realm` | Payload Format |
|:--------|:-----------------------------------------------------------|
| `0x00` | [Version 1 `hop_data`](#version-1-hop_data-payload-format) |
| `0x01` | TLV payload format (to be specified) |
The following `realm`s are defined:

| `realm` | Payload Format | `num_frames`
|:--------|:-----------------------------------------------------------|:---
| `0x00` | [`legacy_hop_data`](#legacy-hop_data-payload-format) | 1
| `0x01` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 1
| `0x02` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 2
| `0x03` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 3
| `0x04` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 4
| `0x05` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 5
| `0x06` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 6
| `0x07` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 7
| `0x08` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 8
| `0x09` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 9
| `0x0a` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 10
| `0x0b` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 11
| `0x0c` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 12
| `0x0d` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 13
| `0x0e` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 14
| `0x0f` | [`tlv_hop_data`](#tlv_hop_data-payload-format) | 15

Using the `hop_payload` field, the origin node is able to precisely specify the path and structure of the HTLCs forwarded at each hop.
As the `hop_payload` is protected under the packet-wide HMAC, the information it contains is fully authenticated with each pair-wise relationship between the HTLC sender (origin node) and each hop in the path.

Using this end-to-end authentication, each hop is able to cross-check the HTLC parameters with the `hop_payload`'s specified values and to ensure that the sending peer hasn't forwarded an ill-crafted HTLC.

## Version 1 `hop_data` Payload Format
## `legacy_hop_data` Payload Format

The version 1 `hop_data` payload format has realm `0x00`, and MUST use a single `frame` to encode the payload.
The original `hop_data` payload format has realm `0x00`, and MUST use a single `frame` to encode the payload.

1. type: `per_hop`
1. type: `legacy_per_hop`
2. data:
* [`8`:`short_channel_id`]
* [`8`:`amt_to_forward`]
Expand Down Expand Up @@ -260,13 +270,44 @@ Field descriptions:
`outgoing_cltv_value`, whether it is the final node or not, to avoid
leaking its position in the route.

* `padding`: This field is for future use and also for ensuring that future non-0-`realm`
`per_hop`s won't change the overall `hops_data` size.
* `padding`: This field is for future use.

When forwarding HTLCs, nodes MUST construct the outgoing HTLC as specified within
`per_hop` above; otherwise, deviation from the specified HTLC parameters
`legacy_per_hop` above; otherwise, deviation from the specified HTLC parameters
may lead to extraneous routing failure.

## `tlv_hop_data` Payload Format

This is a more flexible format, which avoids the redundant
`short_channel_id` field for the final node. The range of `realm`
values also allows use of more than one frame.

1. tlv: `tlv_hop_data`
2. types:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than unroll these fields and take up 4 types, we can instead have them all packed under a single type. All the fields are fixed length, so there's no additional signalling overhead and we also save a few bytes as well. When allocating space in the TLV namespace we should prefer to pack several types into one, if all the types are related to a discrete use case. This model promotes more efficient utilization of the payload space.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well like we discussed in the IRC meeting some of the types would be omitted, for example in the last hop_payload. So while I agree that we may avoid defining types that must always be used in combination, we should also keep in mind that swapping out or omitting individual types is also an important feature here.

1. type: 2 (`amt_to_forward`)
2. data:
* [`integer`:`amt_to_forward`]
1. type: 4 (`outgoing_cltv_value`)
2. data:
* [`integer`:`outgoing_cltv_value`]
1. type: 6 (`short_channel_id`)
2. data:
* [`8`:`short_channel_id`]

### Requirements

The writer:
- MUST include `amt_to_forward` and `outgoing_cltv_value` for every node.
- MUST include `short_channel_id` for every non-final node.
- MUST NOT include `short_channel_id` for the final node.

The reader:
- MUST return an error if `amt_to_forward` or `outgoing_cltv_value` are not present.
- MUST return an error if it is not the final node and `short_channel_id` is not present.

The requirements for the contents of these fields are specified
[above](#legacy-hop_data-payload-format).

# Accepting and Forwarding a Payment

Once a node has decoded the payload it either accepts the payment locally, or forwards it to the peer indicated as the next hop in the payload.
Expand Down Expand Up @@ -411,8 +452,8 @@ following operations:

- The _rho_-key and _mu_-key are generated using the hop's shared secret.
- The `hops_data` field is right-shifted by `hop_payload_len` bytes, discarding the last `hop_payload_len` bytes that exceed its 1300-byte size.
- The payload for the hop is serialized into that hop's `raw_payload`, using the desired format, and the `num_frames_and_realm` is set accordingly.
- The `num_frames_and_realm`, `raw_payload`, `padding` and `HMAC` are copied into the first `hop_payload_len` bytes of the `hops_data`, i.e., the bytes that were just shifted in.
- The payload for the hop is serialized into that hop's `raw_payload`, using the desired format, and the `realm` is set accordingly.
- The `realm`, `raw_payload`, `padding` and `HMAC` are copied into the first `hop_payload_len` bytes of the `hops_data`, i.e., the bytes that were just shifted in.
- The _rho_-key is used to generate 1300 bytes of pseudo-random byte stream which is then applied, with `XOR`, to the `hops_data` field.
- If this is the last hop, i.e. the first iteration, then the tail of the `hops_data` field is overwritten with the routing information `filler` (see [Filler Generation](#filler-generation)).
- The next HMAC is computed (with the _mu_-key as HMAC-key) over the concatenated `hops_data` and associated data.
Expand Down Expand Up @@ -544,7 +585,7 @@ next hop is extracted.
To do so, the processing node copies the `hops_data` field, appends `20*FRAME_SIZE` `0x00`-bytes,
generates `1300 + 20*FRAME_SIZE` pseudo-random bytes (using the _rho_-key), and applies the result
,using `XOR`, to the copy of the `hops_data`.
The first byte of the `hops_data` corresponds to the `num_frames_and_realm` field in the `hop_payload`, which can be decoded to get the `num_frames` and `realm` fields that indicate how many frames are to be parsed and how the `raw_payload` should be interpreted.
The first byte of the `hops_data` corresponds to the `realm` field in the `hop_payload`, which can be decoded to get the `num_frames` field that indicate how many frames are to be parsed and how the `raw_payload` should be interpreted.
The first `num_frames*FRAME_SIZE` bytes of the `hops_data` are the `hop_payload` field used for the the decoding hop.
The next 1300 bytes are the `hops_data` for the outgoing packet destined for the next hop.

Expand Down Expand Up @@ -869,6 +910,11 @@ The channel from the processing node has been disabled.

The CLTV expiry in the HTLC is too far in the future.

1. type: PERM|22 (`required_tlv_missing`)
2. `var_int`:`type`

The `tlv` field was missing a required value of type `type`.

### Requirements

An _erring node_:
Expand Down