Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fatxpool: proper handling of priorities when mempool is full #6647

Open
wants to merge 31 commits into
base: master
Choose a base branch
from

Conversation

michalkucharczyk
Copy link
Contributor

@michalkucharczyk michalkucharczyk commented Nov 26, 2024

Higher-priority transactions can now replace lower-priority transactions even when the internal tx_mem_pool is full.

Notes for reviewers:

  • The tx_mem_pool now maintains information about transaction priority. Although tx_mem_pool itself is stateless, transaction priority is updated after submission to the view. An alternative approach could involve validating transactions at the at block, but this is computationally expensive. To avoid additional validation overhead, I opted to use the priority obtained from runtime during submission to the view. This is the rationale behind introducing the SubmitOutcome struct, which synchronously communicates transaction priority from the view to the pool. This results in a very brief window during which the transaction priority remains unknown - those transaction are not taken into consideration while dropping takes place. In the future, if needed, we could update transaction priority using view revalidation results to keep this information fully up-to-date (as priority of transaction may change with chain-state evolution).
  • When tx_mem_pool becomes full (an event anticipated to be rare), transaction priority must be known to perform priority-based removal. In such cases, the most recent block known is utilized for validation. I think that speculative submission to the view and re-using the priority from this submission would be an unnecessary complication.
  • Once the priority is determined, lower-priority transactions whose cumulative size meets or exceeds the size of the new transaction are collected to ensure the pool size limit is not exceeded.
  • Transaction removed from tx_mem_pool , also needs to be removed from all the views with appropriate event (which is done by remove_transaction_subtree). To ensure complete removal, the PendingTxReplacement struct was re-factored to more generic PendingPreInsertTask (introduced in fatxpool: handling limits and priorities improvements #6405) which covers removal and submssion of transaction in the view which may be potentially created in the background. This is to ensure that removed transaction will not re-enter to the newly created view.

Closes: #5809

@michalkucharczyk michalkucharczyk force-pushed the mku-fatxpool-mempool-priorities-at-limits branch from 724c102 to 4338671 Compare November 26, 2024 12:52
@michalkucharczyk michalkucharczyk force-pushed the mku-fatxpool-mempool-priorities-at-limits branch from 04ac71a to 4f767e5 Compare December 4, 2024 10:03
Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a first pass, only stylistic nits. But will do one more pass.

@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12669113384
Failed job name: cargo-clippy

@michalkucharczyk michalkucharczyk marked this pull request as ready for review January 8, 2025 12:03
@michalkucharczyk michalkucharczyk added T0-node This PR/Issue is related to the topic “node”. R0-silent Changes should not be mentioned in any release notes labels Jan 8, 2025
@michalkucharczyk michalkucharczyk requested a review from a team January 8, 2025 21:35
@michalkucharczyk
Copy link
Contributor Author

/cmd prdoc --bump minor --audience node_dev

prdoc/pr_6647.prdoc Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes T0-node This PR/Issue is related to the topic “node”.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fatxpool: improve support for priority transactions
2 participants