Prepare subspace-farmer-components
to retrieve pieces in batches
#3132
+247
−170
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This introduces
PieceGetter::get_pieces()
(with dummy implementation that just callsPieceGetter::get_piece()
internally) and preparessubspace-farmer-components
to use it.This prepares lower-level part of the farmer implementation, next steps include properly implementing
PieceGetter::get_pieces()
inFarmerPieceGetter
andClusterPieceGetter
to fully utilize new capabilities and then switching piece cache sync toPieceGetter::get_pieces()
.Eventually we'll make similar changes in
subspace-service
and upcoming data retrieval application.I'm also considering optimizing
PieceGetter::get_piece()
to take advantage of already connected peers if there remains a use case for it after all this, though I'll need to check how much sense there is since we can just call with a single piece index, though I realize we'll have a bunch of extra bookkeeping there that might be too heavy for the use case.Note that in
recover_missing_piece
implementation is not of the highest possible efficiency (it waits for one batch to complete before starting to download more pieces), but that is a conscious decision to simplify implementation given that requiring reconstruction is an exceptional situation to begin with and should be a very cold code path overall, it really shouldn't happen on any real network in practice.Code contributor checklist: