Merge remote-tracking branch 'origin/master' into large_channel_support

# Conflicts:
#	01-messaging.md
#	02-peer-protocol.md
#	09-features.md

.aspell.en.pws

@@ -358,6 +358,7 @@ optimizations
 structs
 CompactSize
 encodings
+remotekey
 bigsize
 BigSize
 namespaces
@@ -370,3 +371,4 @@ tlvs
 snprintf
 GitHub
 IRC
+bitmasks

.copy-edit-stylesheet-checklist.md

@@ -1,6 +1,6 @@
 # Basic checklist/stylesheet used for copy editing BOLTs
 
-Contributions should comply with this checklist/stylsheet to maintain correct, clear, consistent, and concise BOLTS. 
+Contributions should comply with this checklist/stylesheet to maintain correct, clear, consistent, and concise BOLTS. 
 
   - spelling
     - run `tools/spellcheck.sh --check [0-9][0-9]-*.md`

01-messaging.md

@@ -108,7 +108,11 @@ messages, a `tlv_stream` is typically placed after all currently defined fields.
 
 The `type` is a varint encoded using the BigSize format. It functions as a
 message-specific, 64-bit identifier for the `tlv_record` determining how the
-contents of `value` should be decoded.
+contents of `value` should be decoded. `type` identifiers below 2^16 are
+reserved for use in this specification. `type` identifiers greater than or equal
+to 2^16 are available for custom records. Any record not defined in this
+specification is considered a custom record. This includes experimental and
+application-specific messages.
 
 The `length` is a varint encoded using the BigSize format signaling the size of
 `value` in bytes.
@@ -121,6 +125,13 @@ according to the message-specific format determined by `type`.
 The sending node:
  - MUST order `tlv_record`s in a `tlv_stream` by monotonically-increasing `type`.
  - MUST minimally encode `type` and `length`.
+ - When defining custom record `type` identifiers:
+   - SHOULD pick random `type` identifiers to avoid collision with other
+     custom types.
+   - SHOULD pick odd `type` identifiers when regular nodes should ignore the
+     additional data.
+   - SHOULD pick even `type` identifiers when regular nodes should reject the
+     full tlv stream containing the custom record.
  - SHOULD NOT use redundant, variable-length encodings in a `tlv_record`.
 
 The receiving node:
@@ -158,7 +169,7 @@ structs, should do so by defining the encoding such that the object is
 serialized within a single `tlv_record`. The uniqueness constraint, among other
 things, enables the following optimizations:
  - canonical ordering is defined independent of the encoded `value`s.
- - canonical ordering can be known at compile-time, rather that being determined
+ - canonical ordering can be known at compile-time, rather than being determined
    dynamically at the time of encoding.
  - verifying canonical ordering requires less state and is less-expensive.
  - variable-size fields can reserve their expected size up front, rather than
@@ -215,14 +226,14 @@ The following convenience types are also defined:
 
 Once authentication is complete, the first message reveals the features supported or required by this node, even if this is a reconnection.
 
-[BOLT #9](09-features.md) specifies lists of features. Each feature is generally represented by 2 bits. The least-significant bit is numbered 0, which is _even_, and the next most significant bit is numbered 1, which is _odd_.
+[BOLT #9](09-features.md) specifies lists of features. Each feature is generally represented by 2 bits. The least-significant bit is numbered 0, which is _even_, and the next most significant bit is numbered 1, which is _odd_.  For historical reasons, features are divided into global and local feature bitmasks.
 
 The `features` field MUST be padded to bytes with 0s.
 
 1. type: 16 (`init`)
 2. data:
-   * [`u16`:`ignorelen`]
-   * [`ignorelen*byte`:`ignore`]
+   * [`u16`:`gflen`]
+   * [`gflen*byte`:`globalfeatures`]
    * [`u16`:`flen`]
    * [`flen*byte`:`features`]
 
@@ -231,9 +242,9 @@ The `features` field MUST be padded to bytes with 0s.
 
 The sending node:
   - MUST send `init` as the first Lightning message for any connection.
-  - MUST set `ignorelen` to zero.
   - MUST set feature bits as defined in [BOLT #9](09-features.md).
   - MUST set any undefined feature bits to 0.
+  - SHOULD NOT set features greater than 13 in `globalfeatures`.
   - SHOULD use the minimum length required to represent the `features` field.
 
 The receiving node:
@@ -246,7 +257,8 @@ The receiving node:
 
 #### Rationale
 
-There used to be two feature bitfields here, but the first is now ignored.
+There used to be two feature bitfields here, but for backwards compatibility they're now
+combined into one.
 
 This semantic allows both future incompatible changes and future backward compatible changes. Bits should generally be assigned in pairs, in order that optional features may later become compulsory.
 
@@ -416,8 +428,8 @@ function that takes a `uint64` value `x` and produces:
 Here `+` denotes concatenation and `be16`, `be32`, and `be64` produce a
 big-endian encoding of the input for 16, 32, and 64-bit integers, respectively.
 
-A value is said to be _minimally encoded_ if it could have been encoded using a
-smaller representation. For example, a BigSize encoding that occupies 5 bytes
+A value is said to be _minimally encoded_ if it could not be encoded using
+fewer bytes. For example, a BigSize encoding that occupies 5 bytes
 but whose value is less than 0x10000 is not minimally encoded. All values
 decoded with BigSize should be checked to ensure they are minimally encoded.
 
@@ -862,7 +874,7 @@ failure:
 
 ## References
 
-1. <a id="reference-2">http://www.unicode.org/charts/PDF/U2600.pdf</a>
+1. <a id="reference-1">http://www.unicode.org/charts/PDF/U2600.pdf</a>
 
 ## Authors
 

02-peer-protocol.md

@@ -6,7 +6,7 @@ operation, and closing.
 # Table of Contents
 
   * [Channel](#channel)
-    * [Definition of `channel_id`](#definition-of-channel-id)
+    * [Definition of `channel_id`](#definition-of-channel_id)
     * [Channel Establishment](#channel-establishment)
       * [The `open_channel` Message](#the-open_channel-message)
       * [The `accept_channel` Message](#the-accept_channel-message)
@@ -365,6 +365,12 @@ This message introduces the `channel_id` to identify the channel. It's derived f
 
 #### Requirements
 
+Both peers:
+  - if `option_static_remotekey` was negotiated:
+    - `option_static_remotekey` applies to all commitment transactions
+  - otherwise:
+    - `option_static_remotekey` does not apply to any commitment transactions
+
 The sender MUST set:
   - `channel_id` by exclusive-OR of the `funding_txid` and the `funding_output_index` from the `funding_created` message.
   - `signature` to the valid signature, using its `funding_pubkey` for the initial commitment transaction, as defined in [BOLT #3](03-transactions.md#commitment-transaction).
@@ -376,6 +382,12 @@ The recipient:
   - on receipt of a valid `funding_signed`:
     - SHOULD broadcast the funding transaction.
 
+#### Rationale
+
+We decide on `option_static_remotekey` at this point when we first have to generate the commitment
+transaction.  Even if a later reconnection does not negotiate this parameter, this channel will continue to use `option_static_remotekey`; we don't support "downgrading".
+This simplifies channel state, particularly penalty transaction handling.
+
 ### The `funding_locked` Message
 
 This message indicates that the funding transaction has reached the `minimum_depth` asked for in `accept_channel`. Once both nodes have sent this, the channel enters normal operating mode.
@@ -388,6 +400,8 @@ This message indicates that the funding transaction has reached the `minimum_dep
 #### Requirements
 
 The sender MUST:
+  - NOT send `funding_locked` unless outpoint of given by `funding_txid` and
+   `funding_output_index` in the `funding_created` message pays exactly `funding_satoshis` to the scriptpubkey specified in [BOLT #3](03-transactions.md#funding-transaction-output).
   - wait until the funding transaction has reached
 `minimum_depth` before sending this message.
   - set `next_per_commitment_point` to the
@@ -396,7 +410,7 @@ transaction, derived as specified in
 [BOLT #3](03-transactions.md#per-commitment-secret-requirements).
 
 A non-funding node (fundee):
-  - SHOULD forget the channel if it does not see the
+  - SHOULD forget the channel if it does not see the correct
 funding transaction after a reasonable timeout.
 
 From the point of waiting for `funding_locked` onward, either node MAY
@@ -410,11 +424,6 @@ funds are at risk. If the fundee were to remember the channel forever, this
 would create a Denial of Service risk; therefore, forgetting it is recommended
 (even if the promise of `push_msat` is significant).
 
-#### Future
-
-An SPV proof could be added and block hashes could be routed in separate
-messages.
-
 ## Channel Close
 
 Nodes can negotiate a mutual close of the connection, which unlike a
@@ -835,7 +844,7 @@ Invalid amounts are a clear protocol violation and indicate a breakdown.
 If a node did not accept multiple HTLCs with the same payment hash, an
 attacker could probe to see if a node had an existing HTLC. This
 requirement, to deal with duplicates, leads to the use of a separate
-identifier; its assumed a 64-bit counter never wraps.
+identifier; it's assumed a 64-bit counter never wraps.
 
 Retransmissions of unacknowledged updates are explicitly allowed for
 reconnection purposes; allowing them at other times simplifies the
@@ -1118,8 +1127,8 @@ messages are), they are independent of requirements here.
    * [`channel_id`:`channel_id`]
    * [`u64`:`next_commitment_number`]
    * [`u64`:`next_revocation_number`]
-   * [`32*byte`:`your_last_per_commitment_secret`] (option_data_loss_protect)
-   * [`point`:`my_current_per_commitment_point`] (option_data_loss_protect)
+   * [`32*byte`:`your_last_per_commitment_secret`] (option_data_loss_protect,option_static_remotekey)
+   * [`point`:`my_current_per_commitment_point`] (option_data_loss_protect,option_static_remotekey)
 
 `next_commitment_number`: A commitment number is a 48-bit
 incrementing counter for each commitment transaction; counters
@@ -1167,10 +1176,13 @@ The sending node:
   next `commitment_signed` it expects to receive.
   - MUST set `next_revocation_number` to the commitment number of the
   next `revoke_and_ack` message it expects to receive.
-  - if it supports `option_data_loss_protect`:
+  - if `option_static_remotekey` applies to the commitment transaction:
+    - MUST set `my_current_per_commitment_point` to a valid point.
+  - otherwise, if it supports `option_data_loss_protect`:
     - MUST set `my_current_per_commitment_point` to its commitment point for
       the last signed commitment it received from its channel peer (i.e. the commitment_point 
       corresponding to the commitment transaction the sender would use to unilaterally close).
+  - if `option_static_remotekey` applies to the commitment transaction, or the sending node supports `option_data_loss_protect`:
     - if `next_revocation_number` equals 0:
       - MUST set `your_last_per_commitment_secret` to all zeroes
     - otherwise:
@@ -1209,7 +1221,16 @@ A node:
       - SHOULD fail the channel.
 
  A receiving node:
-  - if it supports `option_data_loss_protect`, AND the `option_data_loss_protect`
+  - if `option_static_remotekey` applies to the commitment transaction:
+    - if `next_revocation_number` is greater than expected above, AND
+    `your_last_per_commitment_secret` is correct for that
+    `next_revocation_number` minus 1:
+      - MUST NOT broadcast its commitment transaction.
+      - SHOULD fail the channel.
+    - otherwise:
+	  - if `your_last_per_commitment_secret` does not match the expected values:
+        - SHOULD fail the channel.
+  - otherwise, if it supports `option_data_loss_protect`, AND the `option_data_loss_protect`
   fields are present:
     - if `next_revocation_number` is greater than expected above, AND
     `your_last_per_commitment_secret` is correct for that
@@ -1302,6 +1323,14 @@ is valid. However, this also means the fallen-behind node has revealed this
 fact (though not provably: it could be lying), and the other node could use this to
 broadcast a previous state.
 
+`option_static_remotekey` removes the changing `to_remote` key,
+so the `my_current_per_commitment_point` is unnecessary and thus
+ignored (for parsing simplicity, it remains and must be a valid point,
+however), but the disclosure of previous secret still allows
+fall-behind detection.  An implementation can offer both, however, and
+fall back to the `option_data_loss_protect` behavior if
+`option_static_remotekey` is not negotiated.
+
 # Authors
 
 [ FIXME: Insert Author List ]

03-transactions.md

@@ -411,9 +411,9 @@ committed HTLCs:
 
 ## Key Derivation
 
-Each commitment transaction uses a unique set of keys: `localpubkey` and `remotepubkey`.
+Each commitment transaction uses a unique `localpubkey`, and a `remotepubkey`.
 The HTLC-success and HTLC-timeout transactions use `local_delayedpubkey` and `revocationpubkey`.
-These are changed for every transaction based on the `per_commitment_point`.
+These are changed for every transaction based on the `per_commitment_point`, with the exception of `remotepubkey` if `option_static_remotekey` is negotiated.
 
 The reason for key change is so that trustless watching for revoked
 transactions can be outsourced. Such a _watcher_ should not be able to
@@ -426,8 +426,9 @@ avoid storage of every commitment transaction, a _watcher_ can be given the
 the scripts required for the penalty transaction; thus, a _watcher_ need only be
 given (and store) the signatures for each penalty input.
 
-Changing the `localpubkey` and `remotepubkey` every time ensures that commitment
-transaction ID cannot be guessed; every commitment transaction uses an ID
+Changing the `localpubkey` every time ensures that commitment
+transaction ID cannot be guessed except in the trivial case where there is no
+`to_local` output, as every commitment transaction uses an ID
 in its output script. Splitting the `local_delayedpubkey`, which is required for
 the penalty transaction, allows it to be shared with the _watcher_ without
 revealing `localpubkey`; even if both peers use the same _watcher_, nothing is revealed.
@@ -441,14 +442,13 @@ For efficiency, keys are generated from a series of per-commitment secrets
 that are generated from a single seed, which allows the receiver to compactly
 store them (see [below](#efficient-per-commitment-secret-storage)).
 
-### `localpubkey`, `remotepubkey`, `local_htlcpubkey`, `remote_htlcpubkey`, `local_delayedpubkey`, and `remote_delayedpubkey` Derivation
+### `localpubkey`, `local_htlcpubkey`, `remote_htlcpubkey`, `local_delayedpubkey`, and `remote_delayedpubkey` Derivation
 
 These pubkeys are simply generated by addition from their base points:
 
 	pubkey = basepoint + SHA256(per_commitment_point || basepoint) * G
 
 The `localpubkey` uses the local node's `payment_basepoint`;
-the `remotepubkey` uses the remote node's `payment_basepoint`;
 the `local_htlcpubkey` uses the local node's `htlc_basepoint`;
 the `remote_htlcpubkey` uses the remote node's `htlc_basepoint`;
 the `local_delayedpubkey` uses the local node's `delayed_payment_basepoint`;
@@ -459,6 +459,19 @@ secrets are known (i.e. the private keys corresponding to `localpubkey`, `local_
 
     privkey = basepoint_secret + SHA256(per_commitment_point || basepoint)
 
+### `remotepubkey` Derivation
+
+If `option_static_remotekey` is negotiated the `remotepubkey` is simply the
+remote node's `payment_basepoint`, otherwise it is calculated as above using
+the remote node's `payment_basepoint`.
+
+The simplified derivation means that a node can spend a commitment
+transaction even if it has lost data and doesn't know the
+corresponding `per_commitment_point`.  A watchtower could correlate
+transactions given to it which only have a `to_remote` output if it
+sees one of them onchain, but such transactions do not need any
+enforcement and should not be handed to a watchtower.
+
 ### `revocationpubkey` Derivation
 
 The `revocationpubkey` is a blinded key: when the local node wishes to create a new