Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove txs from txpool without dependencies #868

Closed
Voxelot opened this issue Dec 23, 2022 · 2 comments
Closed

Remove txs from txpool without dependencies #868

Voxelot opened this issue Dec 23, 2022 · 2 comments
Assignees
@AurelienFT
Labels
bug Something isn't working fuel-txpool

Comments

@Voxelot
Copy link
Member

Voxelot commented Dec 23, 2022

When a block is produced or imported, the associated txs need to be removed from the pool. However, our current removal method assumes that the tx and all it's dependencies should be removed. This is problematic as the dependent txs may be unnecessarily removed from the txpool.

Expected behavior:

  • when a block is imported
    • the txs that were successfully process should be removed and their dependencies should be kept
    • txs that were skipped should be removed along with any dependencies
  • when transactions are selected for block production
    • they should be removed without removing dependencies, but the depth of the dependencies should be updated
    • any skipped transactions from block production should also remove dependents in the txpool, although this should be handled by the same mechanism as block import
@Voxelot Voxelot mentioned this issue Dec 23, 2022
Merged
@xgreenx xgreenx added the bug Something isn't working label Jan 31, 2023
@xgreenx
Copy link
Collaborator

xgreenx commented Aug 16, 2023

Related to #1229

@xgreenx xgreenx mentioned this issue Jun 14, 2024
Closed
This was referenced Sep 4, 2024
Closed
Merged
AurelienFT added a commit that referenced this issue Oct 3, 2024
@AurelienFT @xgreenx
TxPool v2 General architecture (#2162)
6111cef 
## Linked Issues/PRs
Closes #2160

## Description

This PR contains a new way to organize and store transactions. This new
architecture have multiple benefits :
- Transactions are now organized as a graph and so can easily remove a
branch fixing this kind of problems
#2020
- Storage, collision detection and selections are well split and can be
viewed as independent mechanisms that are plugged to the pool. This
allow to fix this : #1961
and reduce potential related issues in the future.
- Transactions selection is still basic (only ratio gas/tip) but already
address : #868 (for the
selection part) and is highly customizable for next usages.
- Usage of `dependencies` and `dependents` transactions everywhere (not
a mix with parents etc). `dependencies` is for transaction that need to
be included **before** us and `dependents` is for transaction that need
to be included **after** us.
- Reduced the number of caches to maintain to avoid desync between
caches and storages
- Better separation of concerns to have a clearer code in the `Pool`
structure
- Changed the priority over collision by computing the average ratio of
all collision instead of having a better ratio than all collision (idea
from @xgreenx )
- Unwrap is forbidden
- Avoid Arc and have a storage place that can give references on-demand.
- Highly documented
- All the previous tests of the pool passes

### Changes in code logic from TxPool v1
- The verifications are performed in a new order specified in
#2186. The goal is to avoid
making the computation heavy work if the simple checks aren't valid. In
this new version we also ensure that verifications are done in order by
having wrapper type around each step to allow only one verification
path.
- The insertion is performed in a separate thread pool, the goal is to
not block the pool on any verifications/insertions and to manage the
ressources we allocate to these works
- The insertion rules and conditions has change to the following :
- A transaction with dependencies can collide only with one other
transaction
- A transaction without dependencies can collide with multiple
transaction
- Rules to free up space for new transaction
- If a transaction is colliding with another verify if deleting the
colliding transaction and dependents subtree is enough otherwise refuses
the tx
- If a transaction is dependent and not enough space, don't accept
transaction
- If a transaction is executable, try to free has much space used by
less profitable transactions as possible in the pool to include it

- New limits on the size of the pool : max_pool_bytes_size and
max_pool_gas


Some parts of this refactoring can still be improved : 
- Keep not includable transactions in a side buffer
- Benchmarks
- Metrics
- High level documentation (drawings, general explanations) 

Ideas that could be added but maybe overkill : 
- Generic cache structure that is linked to the graph pool somehow, and
automatically update caches when a transaction is included/removed to
avoid cache desync.

## Checklist
- [x] Breaking changes are clearly marked as such in the PR description
and changelog
- [x] New behavior is reflected in tests
- [x] [The specification](https://github.com/FuelLabs/fuel-specs/)
matches the implemented behavior (link update PR if changes are needed)

### Before requesting review
- [x] I have reviewed the code myself
- [ ] I have created follow-up issues caused by this PR and linked them
here

---------

Co-authored-by: Green Baneling <XgreenX9999@gmail.com>
@xgreenx xgreenx mentioned this issue Oct 8, 2024
Merged
5 tasks
AurelienFT added a commit that referenced this issue Oct 8, 2024
@AurelienFT @xgreenx
Linking TxPoolV2 in place of V1 (#2263)
328b42c 
Closes #2020
Closes #1961
Closes #868
Closes #2185
Closes #2186
Closes #2255
Closes #2160
Closes #1967
Closes #2187

## Description
Remove code of the previous txpool and connect all modules to this one

TODO: Improve description @AurelienFT 

## Checklist
- [x] Breaking changes are clearly marked as such in the PR description
and changelog
- [x] New behavior is reflected in tests
- [x] [The specification](https://github.com/FuelLabs/fuel-specs/)
matches the implemented behavior (link update PR if changes are needed)

### Before requesting review
- [x] I have reviewed the code myself
- [x] I have created follow-up issues caused by this PR and linked them
here

---------

Co-authored-by: Green Baneling <XgreenX9999@gmail.com>
@xgreenx
Copy link
Collaborator

xgreenx commented Oct 8, 2024

Done in #2263

@xgreenx xgreenx closed this as completed Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AurelienFT AurelienFT

Labels
bug Something isn't working fuel-txpool
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Voxelot @xgreenx @AurelienFT