Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gossip cleanup part 1: Removing reliance on private gossip messages #6869

Merged
merged 29 commits into from
Dec 13, 2023

Commits on Dec 6, 2023

  1. compiler: fix for -O3 errors.

    CI revealed one:
    
    ```
    cc plugins/libplugin-pay.c
    plugins/libplugin-pay.c: In function ‘payment_getroute’:
    plugins/libplugin-pay.c:888:17: error: ‘errstr’ may be used uninitialized [-Werror=maybe-uninitialized]
      888 |                 payment_fail(p, "%s", errstr);
          |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    plugins/libplugin-pay.c:851:21: note: ‘errstr’ was declared here
      851 |         const char *errstr;
          |                     ^~~~~~
    cc1: all warnings being treated as errors
    ```
    
    My local compiler gave another:
    
    ```
    channeld/channeld.c: In function ‘resume_splice_negotiation’:
    channeld/channeld.c:3734:23: error: ‘final_tx’ may be used uninitialized [-Werror=maybe-uninitialized]
     3734 |                 msg = towire_channeld_splice_confirmed_signed(tmpctx, final_tx,
          |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     3735 |                                                               chan_output_index);
          |                                                               ~~~~~~~~~~~~~~~~~~
    channeld/channeld.c:3461:28: note: ‘final_tx’ was declared here
     3461 |         struct bitcoin_tx *final_tx;
          |                            ^~~~~~~~
    cc1: all warnings being treated as errors
    make: *** [Makefile:298: channeld/channeld.o] Error 1
    ```
    
    So fix both.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    dad0022 View commit details
    Browse the repository at this point in the history
  2. common: prepare unit tests for lack of private channels.

    Created a new canned gossmap without private channels, updated tests.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    522295a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    84361ca View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9b34784 View commit details
    Browse the repository at this point in the history
  5. db: add private channel inbound fields

    This will be used for generating route hints rather than private gossip
    endothermicdev authored and rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    5a29dda View commit details
    Browse the repository at this point in the history
  6. gossipd: pass remote private channel update to ld

    and stash in the database.
    
    Rusty: I added the bad gossip message so we would see unknown updates in CI, and made sure we don't send our own generated updates to lightningd.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    endothermicdev authored and rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4dfe403 View commit details
    Browse the repository at this point in the history
  7. gossipd: always send information about our channels' gossip updates.

    Not just when it's a private channel.  This is useful for listpeerchannels in the next patch.
    Most of this is renaming.
    
    It also means that source can be NULL, so move it out of the struct and put it in the message,
    where it logically belongs, and make it an optional field.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    2fa59d9 View commit details
    Browse the repository at this point in the history
  8. listpeerchannels: show gossip updates.

    This is redundant if it's a public channel, but vital if it's not.  Publishing unconditionally makes
    it easier for gossmap: we create a local modification all the time, even if redundant (and we can
    have the actual capacity ceiling accurate in this case, since we know it for local channels).
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    
    Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    cc215c4 View commit details
    Browse the repository at this point in the history
  9. pyln-testing: when waiting for active channel, make sure remote has s…

    …ent fee info.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    c0bc986 View commit details
    Browse the repository at this point in the history
  10. gossmap: insert temporary per-caller flag to turn off private gossip.

    This lets us convert one user at a time.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    9cb6644 View commit details
    Browse the repository at this point in the history
  11. common: generic routine to turn listpeerchannels into gossmap local u…

    …pdates.
    
    This is more thorough than the minimal one required for getroute(), including the feerates
    and cltv deltas.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    adf5758 View commit details
    Browse the repository at this point in the history
  12. plugins/topology: temporary hack to allow per-access private gossmap …

    …suppression.
    
    We have multiple different routines here, and we want to wean them off private
    gossip one at a time.  This converts `getroute`.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    53ffd08 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. plugins/fetchinvoice: use gossmods_from_listpeerchannels instead of p…

    …rivate gossip_store records.
    
    [ Includes use-after-free fix from Alex Myers <alex@endothermic.dev> ]
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    4a396e8 View commit details
    Browse the repository at this point in the history
  2. libplugin-pay: make sure get_gossmap calls are balanced by put_gossmap.

    This will become important when we apply localmods to the gossmap at these points.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    fcc1010 View commit details
    Browse the repository at this point in the history
  3. plugins/pay: use gossmods_from_listpeerchannels instead of private go…

    …ssip_store records.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    2d06738 View commit details
    Browse the repository at this point in the history
  4. common: add tests that json_scan can omit an entire object member.

    We can, but I had a typo and thought we couldn't!
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    dd3b594 View commit details
    Browse the repository at this point in the history
  5. plugins/renepay: add localmods later.

    We will get localmods from gossmods_from_listpeerchannels in the next commit,
    so we need to save routehints to add to that later.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    88b41fe View commit details
    Browse the repository at this point in the history
  6. plugins/renepay: use gossmods_from_listpeerchannels instead of privat…

    …e gossip_store records.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ccec35d View commit details
    Browse the repository at this point in the history
  7. pytest: fix test_zeroconf_open()

    l2 gave us a routehint, but it should have seen l1 as a dead-end.  It
    didn't due to the presence of a channel alias, which was a bug.  We're
    about to fix this, which breaks the test.
    
    Add a dummy node off l1.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    d536eeb View commit details
    Browse the repository at this point in the history
  8. lightningd: permit wrong-alias lookups for private channel_update.

    This is wrong, but we send them for now in zeroconf!
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    08dda8d View commit details
    Browse the repository at this point in the history
  9. lightningd: make listincoming use gossmods_from_listpeerchannels().

    We temporarily use a second gossmap so we can just switch private info off
    for listincoming and not listchannels.
    
    Note that listchannels now uses the local alias (if no scid), so we have
    to change that in the routehint caller.
    
    Since we now *always* use a channel alias in hints if one exists, a
    test broke, so fix that.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    06efdc7 View commit details
    Browse the repository at this point in the history
  10. pytest: remove test waits for routehints.

    Now listincoming sees local channels in listpeerchannels, we don't need these waits.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    5a81edd View commit details
    Browse the repository at this point in the history
  11. topology: don't show private nodes in listnodes.

    This breaks our tests a bit, which assumed we can always see ourselves
    even if we don't have a proper channel.
    
    Usually we would deprecate this first, but it's unlikely to break
    anyone since it's a bit obscure that this worked at all.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    da4ba26 View commit details
    Browse the repository at this point in the history
  12. pytest: fix tests in assumption that listchannels will no longer show…

    … private gossip.
    
    Some can only be changed once that is true, but some can be removed/amended already.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    f6a2210 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    3362f74 View commit details
    Browse the repository at this point in the history
  14. topology: don't show private nodes in listchannels.

    This alters a few remaining tests, as well.
    
    [ Inclused even more test fixes from Alex Myers <alex@endothermic.dev>! ]
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Changelog-Deprecated: JSON-RPC: `listchannels` listing private channels: use listpeerchannels
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    9bf173c View commit details
    Browse the repository at this point in the history
  15. common: gossmap now always ignores private gossip_store messages.

    In the next PR, they'll be removed, but for now all our code doesn't
    want them.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    1d8f94e View commit details
    Browse the repository at this point in the history
  16. topology: deprecate overriding active in listchannels with `listp…

    …eerchannels` info.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Changelog-Deprecated: `listchannels` no longer uses local knowledge to set `active` to false if disconnected.
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    59a0479 View commit details
    Browse the repository at this point in the history
  17. plugins: re-enable listchannels local info in deprecated mode.

    We only deprecated this, we didn't actually remove it.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    406e6d0 View commit details
    Browse the repository at this point in the history