Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package Relay Draft 1 (Single In-Flight Approach) #8

Closed
wants to merge 57 commits into from

Commits on Apr 18, 2023

  1. Configuration menu
    Copy the full SHA
    43fd360 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f154c34 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5d09d97 View commit details
    Browse the repository at this point in the history
  4. [policy] disallow transactions under min relay fee, even in packages

    Avoid adding transactions below min relay feerate because, even if they
    were bumped through CPFP when entering the mempool, we do not have a
    DoS-resistant way of ensuring they always remain bumped.  In the future,
    this rule can be relaxed (e.g. to allow packages to bump 0-fee
    transactions) if we find a way to do so.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    64fc0da View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1f40f8c View commit details
    Browse the repository at this point in the history
  6. [validation] call AcceptSingleTransaction when only 1 package tx left

    Avoid calling PackageMempoolChecks() when there is only 1 transaction.
    Note to reviewers: there is a slight change in the error type returned,
    as shown in the txpackage_tests change.  When a transaction is the last
    one left in the package and its fee is too low, this returns a PCKG_TX
    instead of PCKG_POLICY. This interface is clearer;
    "package-fee-too-low" for 1 transaction would be a bit misleading.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    3639ad0 View commit details
    Browse the repository at this point in the history
  7. [mempool] evict everything below min relay fee in TrimToSize()

    At this point it's not expected that there are any such transactions,
    except from reorgs.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    a781945 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    daf6b81 View commit details
    Browse the repository at this point in the history
  9. [test] raise wallet_abandonconflict -minrelaytxfee settings

    The intention of the test is to set a high -minrelaytxfee
    such that these transactions are rejected in LoadMempool() and
    in CWallet::ResumbitWalletTransactions().
    
    However, while the parent transactions are below minrelaytxfee, they
    each have descendants high enough feerate to bump them past the
    minrelaytxfee (observe that the `assert_greater_than` checks fail
    if given the original amount of 0.0001). These transactions will be kept
    after the mempool persists packages, which is an improvement, but will
    cause the original test to fail.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    569c284 View commit details
    Browse the repository at this point in the history
  10. [mempool] persist packages across restart

    Hold pool.cs the entire time otherwise wallet resubmissions may call
    TrimtoSize() in between loading transactions from disk.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    faf8b23 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    f9b1993 View commit details
    Browse the repository at this point in the history
  12. scripted-diff: rename CheckPackage to IsPackageWellFormed

    -BEGIN VERIFY SCRIPT-
    sed -i 's/CheckPackage(/IsPackageWellFormed(/g' $(git grep -l CheckPackage)
    -END VERIFY SCRIPT-
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    2d0d4c0 View commit details
    Browse the repository at this point in the history
  13. [packages] Packageifier for arbitrary transaction lists

    We cannot require that peers send topologically sorted lists, because we
    cannot check for this property without ensuring we have the same chain
    tip and ensuring we have the full ancestor set. Instead, add the ability
    to handle arbitrarily ordered transaction lists.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    9bfb4a1 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    87e43a5 View commit details
    Browse the repository at this point in the history
  15. [validation] pre-fill missing inputs for txns depending on invalid

    Packageifier calculates the in-package ancestors. We already know
    this tx will fail because it depends on something invalid (specifically
    for a non-policy reason). We also know that it is missing at least one
    input (the tx that did not and will not make it into our mempool). Just
    return missing inputs directly.
    
    Note to reviewers: slight behavior change. If this tx has multiple
    errors, there may be a difference in which one is returned.  For
    example, if the tx has a dust output in addition to relying on
    the invalid tx, we will return TX_MISSING_INPUTS when we previously
    would have returned TX_NOT_STANDARD. This is simply because dust is
    checked earlier within PreChecks().
    
    Add a test that we don't quit *too* early and reject transactions we
    should keep.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    2109d36 View commit details
    Browse the repository at this point in the history
  16. [validation] validate packages by submitting each tx's ancestor sub-p…

    …ackages
    
    This results in the incentive-compatible transactions ending up in our
    mempool. Prior to this commit, if parents within the package relied on
    each other, we could end up (1) accepting a low-feerate child or (2)
    rejecting high-feerate parents. Instead of validating each transaction
    *individually* in turn, validate each one with their in-package ancestor
    set. This means parents with inter-dependencies are validated correctly,
    while we continue using aggregate totals for package feerate.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    9bcefd6 View commit details
    Browse the repository at this point in the history
  17. [policy] allow any ancestor package, not just child-with-unconfirmed-…

    …parents
    
    We can safely allow any ancestor package since Packageifier can handle
    things that are out of order. Remove the check that "all unconfirmed
    parents are present" because, even if a tx is missing inputs, the other
    transactions may be worth validating.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    0c011bc View commit details
    Browse the repository at this point in the history
  18. -- Part 1: Orphan Resolution Module --

    Create TxPackageTracker module, responsible for handling orphan
    resolution. Enable node to retry orphan parent downloading from multiple
    peers who announced the tx. Use preferred peers and avoid overloading
    peers with too many requests.
    
    Making the orphanage more reliable is also necessary because it becomes
    part of the "critical path" for some transactions.  A 0-fee parent
    bumped by a child *must* be relayed via package relay; if the orphanage
    overflows or is churned by a malicious peer, the package would be
    censored from the network.
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    e42d8d9 View commit details
    Browse the repository at this point in the history
  19. [p2p] add tx package tracker

    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    0821236 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    bcb33bb View commit details
    Browse the repository at this point in the history
  21. [txorphange] GetTx by wtxid

    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    718ce4a View commit details
    Browse the repository at this point in the history
  22. [txorphanage] EraseTx by wtxid

    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    60566f4 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    dce6d8b View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    2e72630 View commit details
    Browse the repository at this point in the history
  25. [txorphanage] impose a maximum total size of orphans

    No effect for now.
    TODO: add per-peer limits on orphan memory usage
    glozow committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    49aa386 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2023

  1. Configuration menu
    Copy the full SHA
    9a65983 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2861927 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3df9a46 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8325214 View commit details
    Browse the repository at this point in the history
  5. [refactor] use txpackagetracker for orphan resolution

    No behavior change in this commit.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    c24c133 View commit details
    Browse the repository at this point in the history
  6. [txpackagetracker] delete unused OrphanageAddTx

    Adding/deleting is now strictly handled by TxPackageTracker, and should
    not be exposed to the public.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    44c3725 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e4d7b44 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    b46bafb View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    26d7bdc View commit details
    Browse the repository at this point in the history
  10. [p2p] use all orphan announcers as potential parent sources

    Makes orphan handling more robust. We will also use the orphan
    resolution tracker for requesting ancpkginfo. The timeout in
    p2p_segwit.py has been increased to account for the fact that parents
    are no longer immediately requested.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    6023be6 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    f5b00f3 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    5c7331b View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    945814c View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    3c0e90c View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    424eb15 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    25d3fe4 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    797ed82 View commit details
    Browse the repository at this point in the history
  18. [validation/p2p] separate TxValidationResult and rejects filter for l…

    …ow fees
    
    We wouldn't want to add a low-feerate transaction to m_recent_rejects
    because usually that means we won't give the child a chance
    (`fRejectedParents`); we instead want to fetch this orphan's low-feerate
    parents and potentially accept them.  However, we also shouldn't
    revalidate transactions/packages that have already been rejected.
    Continue to cache low-fee rejections, but separately.
    
    Note: when ephemeral anchor transaction fails due to a missing spender,
    we should similarly put it in the reconsiderable filter.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    f7d6fea View commit details
    Browse the repository at this point in the history
  19. [p2p] respond to getdata(ancpkginfo) requests with ancpkginfo

    Conditions for responding to an ancpkginfo request are identical to
    responding to a tx. That is, we only provide ancpkginfo if we would have
    provided tx data as well.
    (HACKY AND UNSAFE)
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    8164ad2 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    e6656a9 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    0389892 View commit details
    Browse the repository at this point in the history
  22. [p2p] Resolve orphans by requesting ancpkginfo and requesting txdata

    For now, each tx in ancpkginfo is treated like an individual tx
    announcement. When the tx arrives, it is validated individually as well.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    21acc89 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    cc800b3 View commit details
    Browse the repository at this point in the history
  24. [p2p] respond to getpkgtxns with pkgtxns or notfound MSG_PKGTXNS

    FIXME: packagehash for python tests
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    ba5e401 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    852da8e View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    fbe58b5 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    8859429 View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    252ff5a View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    54c4fbe View commit details
    Browse the repository at this point in the history
  30. [p2p] don't inv fee-bumping children to non-pkgrelay peers

    These transactions can be more common with package relay, but
    non-package-relay peers will only waste bandwidth and orphanage space,
    then end up not accepting them. Save their bandwidth by dropping
    announcements of transactions that have below-fee-filter parents.
    glozow committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    efa53b7 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    d682a10 View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    b647166 View commit details
    Browse the repository at this point in the history