Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support large scale for ads mode #610

Merged
merged 22 commits into from
Sep 27, 2024
Merged

support large scale for ads mode #610

merged 22 commits into from
Sep 27, 2024

Conversation

nlgwcy
Copy link
Contributor

@nlgwcy nlgwcy commented Jul 18, 2024

What type of PR is this?

/kind enhancement

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

@kmesh-bot kmesh-bot added the kind/enhancement New feature or request label Jul 18, 2024
@nlgwcy nlgwcy added do-not-merge/work-in-progress and removed kind/enhancement New feature or request labels Jul 18, 2024
@nlgwcy nlgwcy changed the title support lage scale for ads mode support large scale for ads mode Jul 19, 2024
@nlgwcy
support lage scale for ads mode
8253242 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@codecov Codecov
Copy link

codecov bot commented Jul 19, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 50.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 53.12%. Comparing base (0c5498c) to head (6856384).
Report is 28 commits behind head on main.

Files with missing lines Patch % Lines
pkg/bpf/bpf.go 50.00% 2 Missing and 1 partial ⚠️
Files with missing lines Coverage Δ
pkg/bpf/bpf.go 43.60% <50.00%> (+2.53%) ⬆️

... and 3 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20cb2d8...6856384. Read the comment docs.

nlgwcy added 9 commits July 19, 2024 21:41
@nlgwcy
adjust ads bpf map size
5723333 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
outter_map supports non-continuous elastic scaleup
ee29243 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
outter_map supports non-continuous elastic scalein
49d16b6 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
update max_alloced_idx for scale-in
a46840f 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
inner_map_stat.in_outer_map rename
721f6f5 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
map-in-map mng persist
77a40d2 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
map-in-map mng restore
9b00a14 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
support scale up after restore
bd642b2 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
add docs
a8f40d5 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@kmesh-bot kmesh-bot added size/XL and removed size/L labels Jul 20, 2024
@nlgwcy
magic check when restore
8eaf62e 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
reset inner_maps item in inner_map_batch_delete
3945e11 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@kmesh-bot
Copy link
Collaborator

Adding label do-not-merge/contains-merge-commits because PR contains merge commits, which are not allowed in this repository.
Use git rebase to reapply your commits on top of the target branch. Detailed instructions for doing so can be found here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@nlgwcy
fix build warnning
075b752 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
fix conflict
05f2ac9 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
fix issue
d35e205 
Signed-off-by: wuchangye <wuchangye@huawei.com>
hzxuzhonghu
hzxuzhonghu previously approved these changes Sep 27, 2024
View reviewed changes
@hzxuzhonghu
Copy link
Member

/lgtm

@kmesh-bot kmesh-bot added the lgtm label Sep 27, 2024
@hzxuzhonghu
Copy link
Member

/retest

@nlgwcy
fix ci failed
a5141a6 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@nlgwcy
Copy link
Contributor Author

nlgwcy commented Sep 27, 2024

/retest

@nlgwcy
fix panic
ac3dbe6 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@hzxuzhonghu
Copy link
Member

TestRestartReload Still failed

@nlgwcy
remove persist file when uninit
dbe585e 
Signed-off-by: wuchangye <wuchangye@huawei.com>
hzxuzhonghu
hzxuzhonghu reviewed Sep 27, 2024
View reviewed changes
bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.c
@@ -1603,6 +1603,8 @@ void deserial_uninit(bool persist)
{
if (persist)
inner_map_mng_persist();
else
remove(MAP_IN_MAP_MNG_PERSIST_FILE_PATH);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hha

hzxuzhonghu
hzxuzhonghu reviewed Sep 27, 2024
View reviewed changes
pkg/utils/test/bpf_map.go Outdated
@@ -31,6 +31,7 @@ import (
type CleanupFn func()

func InitBpfMap(t *testing.T, config options.BpfConfig) (CleanupFn, *bpf.BpfLoader) {
bpf.CleanupBpfMap()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this? Any test forget to cleanup before exit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry... I forget to remove, just for test.

@nlgwcy
fix deadloop in inner_map_mng_restore_by_persist_stat
6856384 
Signed-off-by: wuchangye <wuchangye@huawei.com>
@hzxuzhonghu
Copy link
Member

/lgtm

@kmesh-bot kmesh-bot added the lgtm label Sep 27, 2024
@hzxuzhonghu
Copy link
Member

/approve

@kmesh-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kmesh-bot kmesh-bot merged commit 608e998 into kmesh-net:main Sep 27, 2024
9 checks passed
@kmesh-bot
Copy link
Collaborator

In response to a cherrypick label: #610 failed to apply on top of branch "release-0.5":

Applying: support lage scale for ads mode
Using index info to reconstruct a base tree...
M	bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.c
M	bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.h
Falling back to patching base and 3-way merge...
Auto-merging bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.h
Auto-merging bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.c
CONFLICT (content): Merge conflict in bpf/deserialization_to_bpf_map/deserialization_to_bpf_map.c
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 support lage scale for ads mode
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

@kmesh-bot
Copy link
Collaborator

In response to a cherrypick label: new issue created for failed cherrypick: #907

@kmesh-bot kmesh-bot mentioned this pull request Sep 27, 2024
Closed
kmesh-bot added a commit that referenced this pull request Sep 27, 2024
@kmesh-bot
Merge pull request #909 from nlgwcy/release-0.5
a49b6e8 
cherry-pick #610 to Release 0.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hzxuzhonghu hzxuzhonghu hzxuzhonghu left review comments

@supercharge-xsy supercharge-xsy Awaiting requested review from supercharge-xsy

Assignees

@hzxuzhonghu hzxuzhonghu

Labels
approved cherrypick/release-0.5 do-not-merge/contains-merge-commits lgtm size/XL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nlgwcy @kmesh-bot @hzxuzhonghu