Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

buffers data shreds to make larger erasure coded sets (bp #15849) #16074

Merged
merged 2 commits into from
Mar 23, 2021

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Mar 23, 2021

This is an automatic backport of pull request #15849 done by Mergify.

Cherry-pick of 4f82b89 has failed:

On branch mergify/bp/v1.6/pr-15849
Your branch is up to date with 'origin/v1.6'.

You are currently cherry-picking commit 4f82b897b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   core/benches/shredder.rs
	modified:   core/src/broadcast_stage.rs
	modified:   core/src/broadcast_stage/broadcast_utils.rs
	modified:   core/src/broadcast_stage/standard_broadcast_run.rs
	modified:   core/src/shred_fetch_stage.rs
	modified:   ledger/src/blockstore.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   ledger/src/shred.rs

To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

@mergify mergify bot added conflicts automerge Merge this Pull Request automatically once CI passes labels Mar 23, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Mar 23, 2021

automerge label removed due to a CI failure

@mergify mergify bot removed the automerge Merge this Pull Request automatically once CI passes label Mar 23, 2021
@behzadnouri
Copy link
Contributor

@Mergifyio rebase

Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.

(cherry picked from commit 4f82b89)

# Conflicts:
#	ledger/src/shred.rs
@jackcmay jackcmay force-pushed the mergify/bp/v1.6/pr-15849 branch from 07e348f to 222feba Compare March 23, 2021 17:02
@mergify
Copy link
Contributor Author

mergify bot commented Mar 23, 2021

Command rebase: success

Branch has been successfully rebased

@mergify mergify bot added automerge Merge this Pull Request automatically once CI passes and removed automerge Merge this Pull Request automatically once CI passes labels Mar 23, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Mar 23, 2021

automerge label removed due to a CI failure

@mergify mergify bot added the automerge Merge this Pull Request automatically once CI passes label Mar 23, 2021
@codecov
Copy link

codecov bot commented Mar 23, 2021

Codecov Report

Merging #16074 (818775c) into v1.6 (a04ca03) will increase coverage by 0.0%.
The diff coverage is 88.8%.

@@           Coverage Diff           @@
##             v1.6   #16074   +/-   ##
=======================================
  Coverage    79.9%    79.9%           
=======================================
  Files         409      409           
  Lines      107349   107443   +94     
=======================================
+ Hits        85779    85862   +83     
- Misses      21570    21581   +11     

@mergify mergify bot merged commit 9d37a33 into v1.6 Mar 23, 2021
@mergify mergify bot deleted the mergify/bp/v1.6/pr-15849 branch March 23, 2021 18:23
@brooksprumo brooksprumo mentioned this pull request Aug 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
automerge Merge this Pull Request automatically once CI passes conflicts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant