Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autoclean for giant nodes #7298

Merged

Commits on Jun 19, 2024

  1. lightningd: don't load all forwardings into memory at once.

    Use modern-style iterators: this can be huge and we will run out of
    memory!
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    e31ee43 View commit details
    Browse the repository at this point in the history
  2. autoclean: remove autocleaninvoice command.

    Changelog-Removed: JSON-RPC: `autocleaninvoice` command (deprecated v22.11, EOL v24.02)
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a4b95e6 View commit details
    Browse the repository at this point in the history
  3. autoclean: use a pointer for our global clean_info.

    We're about to make this a more complex struct, so introducing a
    new_clean_info() function unifies the code paths.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    625d703 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. autoclean: rework logic to clarify subsystems.

    There are really three subsystems: invoices, forwards and sendpays,
    each of which has two variants we care about (successes and failures).
    If we split the code that way, we can extract the core differences in
    each of these cases and share most of the logic.
    
    It's a bit awkward to iterate over each "subsystem" in the JSON
    parameter sense, so we have some iteration code to do that where we
    need to.
    
    The result is going to be much easier to paginate!
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    a13eb9b View commit details
    Browse the repository at this point in the history
  2. libplugin: support filtering of outgoing commands.

    This is crude, handing a raw JSON string, but it works.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    8b4eae9 View commit details
    Browse the repository at this point in the history
  3. libplugin: add plugin_option_dev_dynamic (our first dynamic dev optio…

    …n coming).
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    bdc154d View commit details
    Browse the repository at this point in the history
  4. autoclean: use filter to reduce processing.

    We can filter down to only the list* fields we need.  In the case of a
    node with 1M forwards, this reduces listforwards time from 5 seconds
    to 4 seconds.  It will also reduce memory consumption.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    3b86e6c View commit details
    Browse the repository at this point in the history
  5. autoclean: call list in easy stages.

    listforwards on a large node can easily run out of memory.  Sip, don't
    gulp!
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    e6cfe91 View commit details
    Browse the repository at this point in the history