Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classify chunked items correctly. Issue #5850 #5852

Merged
merged 1 commit into from
Jul 3, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/libsync/discoveryphase.h
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,10 @@ struct SyncOptions
/** If a confirmation should be asked for external storages */
bool _confirmExternalStorage;

/** The initial un-adjusted chunk size in bytes for chunked uploads
/** The initial un-adjusted chunk size in bytes for chunked uploads, both
* for old and new chunking algorithm, which classifies the item to be chunked
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you say "which classifies the item to be chunked" (also below) that's confusing because it's ambiguous. "which is used to decide whether an item should be chunked" would be clearer.

*
* When dynamic chunk size adjustments are done, this is the
* In chunkingNG, when dynamic chunk size adjustments are done, this is the
* starting value and is then gradually adjusted within the
* minChunkSize / maxChunkSize bounds.
*/
Expand Down
3 changes: 2 additions & 1 deletion src/libsync/owncloudpropagator.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,8 @@ PropagateItemJob *OwncloudPropagator::createJob(const SyncFileItemPtr &item)
return job;
} else {
PropagateUploadFileCommon *job = 0;
if (item->_size > _chunkSize && account()->capabilities().chunkingNg()) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of that, after dynamic chunk has been readjusted to 100MB, this file has been classified as V1, being 50MB in size. With _initialChunkSize being 10MB, it is classified as to be chunked and having high dynamic chunk, being send in one shot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that's correct. Thanks for spotting and fixing this.

if (item->_size > syncOptions()._initialChunkSize && account()->capabilities().chunkingNg()) {
// Item is above _initialChunkSize, thus will be classified as to be chunked
job = new PropagateUploadFileNG(this, item);
} else {
job = new PropagateUploadFileV1(this, item);
Expand Down
6 changes: 5 additions & 1 deletion src/libsync/propagateupload.h
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,11 @@ class PropagateUploadFileV1 : public PropagateUploadFileCommon
int _chunkCount; /// Total number of chunks for this file
int _transferId; /// transfer id (part of the url)

quint64 chunkSize() const { return propagator()->syncOptions()._initialChunkSize; }
quint64 chunkSize() const {
// Old chunking does not use dynamic chunking algorithm, and does not adjusts the chunk size respectively,
// thus this value should be used as the one classifing item to be chunked
return propagator()->syncOptions()._initialChunkSize;
}


public:
Expand Down
1 change: 1 addition & 0 deletions src/libsync/propagateuploadng.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,7 @@ void PropagateUploadFileNG::slotPutFinished()
// the chunk sizes a bit.
quint64 targetSize = (propagator()->_chunkSize + predictedGoodSize) / 2;

// Adjust the dynamic chunk size _chunkSize used for sizing of the item's chunks to be send
propagator()->_chunkSize = qBound(
propagator()->syncOptions()._minChunkSize,
targetSize,
Expand Down