feat: implement parallel operations (#2067) #2109
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ddelgrosso1
added
the
do not merge
|
Here is the summary of changes. You are about to add 3 region tags.
This comment is generated by snippet-bot.
|
product-auto-label
bot
added
size: xl
ddelgrosso1
added
the
owlbot:run
gcf-owl-bot
bot
removed
the
owlbot:run
* feat: implement parallel operations * add more parallel operations * add header to test file * update import of fs/promises * fix pathing on windows, fix mocking of fs promises * add jsdoc headers to class and uploadMulti * add jsdoc comments to remaining functions * update comment wording * add experimental jsdoc tags * feat: add directory generator to performance test framework * clarify variable names and comments * capitalization * wip: transfer manager performance tests * feat: merged in application performance tests (#2100) * fix: fixed many bugs (#2102) * fix: cleaning up bugs * fix: fixed many bugs * fix: more work on transfer manager perf metrics (#2103) * fix: more work on transfer manager perf metrics * fix unit tests for tm * fix: performance test refactoring, comments (#2104) * refactor: refactor constants (#2105) * refactor: refactor constants * bug fixes * bug fixes * bug fixes * linter fixes, download to disk for performance test * rename transfer manager functions * remove callbacks from transfer manager * add more experimental tags, update comments * change signature of downloadManyFiles to accept array of strings or a folder name * linter fix * add transfer manager samples and samples tests Co-authored-by: Sameena Shaffeeullah <shaffeeullah@google.com>
ddelgrosso1
force-pushed
the
transfer-manager
branch
from
53ac5b8 to
6ff1eb5
Compare
|
Warning: This pull request is touching the following templated files:
|
ddelgrosso1
added
the
owlbot:run
gcf-owl-bot
bot
removed
the
owlbot:run
#2110) * feat: add crc32c validation option to chunked download, cleanup naming in perf tests * close file in finally block
ddelgrosso1
removed
the
do not merge
shaffeeullah
approved these changes
Dec 6, 2022
frankyn
reviewed
Dec 6, 2022
|
|
||
| async function uploadManyFilesWithTransferManager() { | ||
| // Uploads the files | ||
| await transferManager.uploadManyFiles([firstFilePath, secondFilePath]); |
There was a problem hiding this comment.
Only nit is samples don't show users how to upload / download a directory similar to Python samples; otherwise LGTM.
danielbankhead
approved these changes
Dec 7, 2022
|
|
||
| export interface DownloadFileInChunksOptions { | ||
| concurrencyLimit?: number; | ||
| chunkSizeBytes?: number; |
There was a problem hiding this comment.
Btw, I appreciate the unit included with the value
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: implement parallel operations
add more parallel operations
add header to test file
update import of fs/promises
fix pathing on windows, fix mocking of fs promises
add jsdoc headers to class and uploadMulti
add jsdoc comments to remaining functions
update comment wording
add experimental jsdoc tags
feat: add directory generator to performance test framework
clarify variable names and comments
capitalization
wip: transfer manager performance tests
feat: merged in application performance tests (feat: merged in application performance tests #2100)
fix: fixed many bugs (fix: fixed many bugs #2102)
fix: cleaning up bugs
fix: fixed many bugs
fix: more work on transfer manager perf metrics (fix: more work on transfer manager perf metrics #2103)
fix: more work on transfer manager perf metrics
fix unit tests for tm
fix: performance test refactoring, comments (fix: performance test refactoring, comments #2104)
refactor: refactor constants (refactor: refactor constants #2105)
refactor: refactor constants
bug fixes
bug fixes
bug fixes
linter fixes, download to disk for performance test
rename transfer manager functions
remove callbacks from transfer manager
add more experimental tags, update comments
change signature of downloadManyFiles to accept array of strings or a folder name
linter fix
add transfer manager samples and samples tests
Co-authored-by: Sameena Shaffeeullah shaffeeullah@google.com
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕