Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce network IOs by amplifying prefetch requests #609

Merged
merged 2 commits into from
Jul 26, 2022

Conversation

jiangliu
Copy link
Collaborator

Reduce backend network IOs by enabling amplification of v6 prefetch requests. For a java image, it could reduce 50% network requests by average.

截屏2022-07-25 下午7 51 12

@anolis-bot
Copy link
Collaborator

@jiangliu , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/15806

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

size,
state.clone(),
));
rt.spawn(async move {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why delegate the fetching data Future into the same tokio runtime rather than directly run to the end of future handle_prefetch_requests . Moreover, it seems the runtime rt only has a thread, so re-generate a future is efficent ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current future acts as a dispatcher, which receives requests from the mpmc queue and dispatch them.
So there may be multiple inflight requests.
Currently two inlfight-requests are allowed.

storage/src/meta/mod.rs Outdated Show resolved Hide resolved
Try to amplify prefetch request for v6 image to reduce network requests.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Limit number of inflight prefetch requests per thread, to give higher
priority to on-demand requests.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/15959

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@jiangliu jiangliu requested a review from changweige July 26, 2022 08:45
Copy link
Contributor

@changweige changweige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@changweige changweige merged commit 0c0b544 into dragonflyoss:master Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants