Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

splicing: Adds the features needed to enable collaborative splicing & resizing of active channels. #6253

Merged
merged 13 commits into from
Jul 31, 2023
Merged

splicing: Adds the features needed to enable collaborative splicing & resizing of active channels. #6253

merged 13 commits into from
Jul 31, 2023

Conversation

ddustin
Copy link
Collaborator

@ddustin ddustin commented May 12, 2023

Note: Github force-closed my previous PR #5675, so I had to open a new one here.

This PR is meant to add splicing to CoreLightning. Splicing enables the resizing of multiple channels in a single on-chain transaction, without any channels being "down" or closed during any part of the process. While enabling the resizing of multiple of your own channels -- splicing also gives your peers (and their peers, and their peers' peers, etc.) an opportunity to dynamically add channels as well.

Not only can participants add their own resizes, due to compatibility with @niftynei's dual funding feature, they can also add channel opens into the same batch. Eventually dual closing will allow all onchain lightning events to be batched into a single bitcoin transaction.

This will also, in time, enable wallets to implement "one balance" apps that allow users to send lightning & on-chain payments without needing to manage two "bitcoin balances" -- with the lowest fees possible.

The code implements splicing spec located here: lightning/bolts#863

My endless gratitude to @niftynei 🙏 who has helped an insane amount with planning, disagnosing, and literally writing parts of the code -- not sure I could have gotten this far without her. Thanks as well to @cdecker, @rustyrussell and everyone who helped me while working on this.

Here are the things I believe need to be done before the PR can be considered ready for a developer-only release

  • Refactor splice_init RPC to take a user's initial PSBT
  • Do byte ownership fee calculations
  • Enable signature ordering logic
  • Add splice commit sigs to inflight
  • Redo inflight process to cache a copy in channeld
  • Add BOLT messages with corresponding commit hashes
  • Change splice RPC to use channel_id instead of peer_id
  • Finish Collaborative transaction building #5287
  • Refactor this code to use Collaborative transaction building #5287
  • Refactor RPC to remove splice_finalized
  • Move wire updates to experimental patches
  • Add #if EXPERIMENTAL && DEVELOPER flags around code
  • Diagnose why the committed_splice_count check fails in handle_peer_commitsig
  • Update channel balance on splice_locked & save to DB
  • Load correct channel balance post splice_lock after restart
  • Test splicing a post-spliced channel
  • Check for TODO items in code
  • Correct balance checks for the l2 code path lisa gist. See check_balances
  • Reconnect logic

@ddustin ddustin requested a review from cdecker as a code owner May 12, 2023 01:20
@ddustin ddustin mentioned this pull request May 12, 2023
Closed
19 tasks
@t-bast t-bast mentioned this pull request May 15, 2023
Closed
rustyrussell
rustyrussell reviewed Jun 1, 2023
View reviewed changes
Copy link
Contributor

@rustyrussell rustyrussell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I haven't reviewed all the fixups, but I got through the actual main commits!

hsmd/hsmd_wire.csv Outdated Show resolved Hide resolved
common/psbt_internal.c Outdated Show resolved Hide resolved
common/psbt_internal.c Outdated Show resolved Hide resolved
common/psbt_internal.c Outdated Show resolved Hide resolved
common/gossip_constants.h Outdated Show resolved Hide resolved
lightningd/channel_control.c Outdated Show resolved Hide resolved
lightningd/lightningd.c Outdated Show resolved Hide resolved
tests/test_db.py Outdated Show resolved Hide resolved
channeld/channeld.c Outdated Show resolved Hide resolved
contrib/startup_regtest.sh Outdated Show resolved Hide resolved
@ddustin ddustin force-pushed the ddustin/splice branch 6 times, most recently from ad722c3 to 0de8092 Compare June 27, 2023 18:42
@ddustin ddustin mentioned this pull request Jun 28, 2023
Closed
This was referenced Jul 6, 2023
Open
Open
@ddustin ddustin force-pushed the ddustin/splice branch 8 times, most recently from 6e40fec to 7101be9 Compare July 11, 2023 16:14
@rustyrussell rustyrussell added this to the v23.08 milestone Jul 13, 2023
@ddustin ddustin force-pushed the ddustin/splice branch 5 times, most recently from e46cd6a to a4ab946 Compare July 19, 2023 20:15
@ddustin ddustin force-pushed the ddustin/splice branch 4 times, most recently from 112fc41 to f63bc51 Compare July 27, 2023 21:43
@litch
Copy link
Contributor

litch commented Jul 28, 2023 •
edited
Loading

I'm testing this in a mixed-nodes environment and it looks like a post-spliced channel is un-detectable by LND. I'm not 100% sure why at this point, and have some logs I can provide on request.

Workflow:

  • Lnd1 is connected to cln1
  • Open a channel between cln1 and cln2
  • Verify that Lnd1 can see the channel between cln1 and cln2 in its describegraph
  • Splice the channel
  • the channel between cln1 and cln2 is now not listed in the describegraph
  • Open a new channel between cln1 and cln2
  • Verify that the new channel between cln1 and cln2 appears in the describegraph, whereas the spliced one does not

Happy to help debug!

EDIT TO ADD:

It looks the newly-created channel eventually does get broadcast. The ChannelAnnounce message wound up getting broadcast an hour after the splice happened. Even with dev-fast-gossip enabled and I restarted the CLN node within that hour window (at about 40 minutes). Unsure if that is related.

In any case: tACK (at least in my cluster of regtest nodes)

@rustyrussell
Copy link
Contributor

You can't advertize a channel less than 6 blocks old, so the "new" channel won't appear for this long. However, implementations are supposed not to discard "closed" channels for 12 blocks, so they old channel id can still be used in the interim...

@rustyrussell
Copy link
Contributor

Rebased against trivial conflict....

ddustin and others added 13 commits July 31, 2023 20:49
@ddustin @rustyrussell
Mac: Update Makefile to work on Mac zsh
c70ff21 
Using the -n flag on echo is non-standard for zsh. printf ‘%s’ accomplishes the equivilent thing.

Changelog-Added: Small fix for Mac OS building

# Conflicts:
#	plugins/Makefile
@ddustin @rustyrussell
Mac: Change format type to use PRIu64 instead of %ld
d0de384 
Changelog-None
@ddustin @rustyrussell
hsmd: Add sign splice command
19f2958 
Changelog-None
@ddustin @rustyrussell
psbt: routine for lining up sigantures in a witness stack
d583cc5 
Changelog-None
@ddustin @rustyrussell
splicing: Add channel state AWAITING_SPLICE
faba7cc 
Update gossip routiens and various other hecks on the channel state to consider AWAITING_SPLICE to be routable and treated similar to CHANNELD_NORMAL.

Small updates to psbt interface

Changelog-None
@ddustin @rustyrussell
psbt: Fix psbt linseraize_input bug
f1dba7a 
Some fields weren’t intitialized causing complex crashes elsewhere.

Changelog-None
@ddustin @rustyrussell
splicing: add DB details for splice HTLCs
49c09b1 
Changelog-None
@ddustin @rustyrussell
interactive-tx: Renaming for clarity and cleaning up
61b5ffe 
New daemon process means we don’t have to deal with gossip, so that gets removed along with error cleanup and a refactoring of how we calculating PDBT diffs.
@ddustin @rustyrussell
channeld: Code to implement splicing
4535fce 
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.

Implement the channeld splicing protocol leveraging the interactivetx protocol.

Implement lightningd’s channel_control to support channeld in its splicing efforts.

Changelog-Added: Added the features to enable splicing & resizing of active channels.
@niftynei @rustyrussell
contrib/startup: ignore db-updates
c1cc088
@niftynei @rustyrussell
inflights: use ctx for making new ones
eaad689 
Also convert everything to an array thingy
@ddustin @rustyrussell
signed types: integer fix
d3e24f2 
integer was missing from the list of type conversions causing CI to fall over

ChangeLog-None
@rustyrussell
CI: Try not running group 2/10 UBSAN in parallel.
c8bf63a 
It's being killed with signal 143, which means docker isn't happy; too much memory consumption?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@rustyrussell
Copy link
Contributor

... and again for CI breakage fix.

@rustyrussell rustyrussell enabled auto-merge (rebase) July 31, 2023 11:20
@rustyrussell rustyrussell merged commit 3974806 into ElementsProject:master Jul 31, 2023
9 of 10 checks passed
@litch litch mentioned this pull request Jul 31, 2023
Merged
@benthecarman benthecarman mentioned this pull request Jul 31, 2023
Closed
@ddustin ddustin mentioned this pull request Aug 1, 2023
Closed
@vnprc vnprc mentioned this pull request Sep 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rustyrussell rustyrussell rustyrussell approved these changes

@niftynei niftynei Awaiting requested review from niftynei

@cdecker cdecker Awaiting requested review from cdecker cdecker is a code owner

@ZmnSCPxj ZmnSCPxj Awaiting requested review from ZmnSCPxj

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v23.08
Development

Successfully merging this pull request may close these issues.
4 participants
@ddustin @litch @rustyrussell @niftynei