Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add code for part peers backward compatible #4101

Merged
merged 8 commits into from
Apr 6, 2022

Conversation

pengweisong
Copy link
Contributor

@pengweisong pengweisong commented Mar 30, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

When persist peers info in the storage, we should remember to fill the data which upgrade from the old version.

How do you solve it?

Move the balance peer info to another system table: balance table.
When start the storaged, we should:

  1. load the balance parts.
  2. load the local parts.
    • If the part already exist in the balance parts, skip it.
    • Else if the part the part not in the part manager, skip it.
  3. load the parts from part manager.
    • If the part already been created, skip it.
    • Will not erase any part in this phase.

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

@pengweisong pengweisong requested a review from critical27 March 30, 2022 17:05
@pengweisong pengweisong added the ready-for-testing PR: ready for the CI test label Mar 30, 2022
@Sophie-Xie Sophie-Xie added this to the v3.1.0 milestone Mar 31, 2022
@Sophie-Xie Sophie-Xie added the cherry-pick-v3.1 PR: need cherry-pick to this version label Apr 1, 2022
@liwenhui-soul
Copy link
Contributor

good~

src/kvstore/Common.h Outdated Show resolved Hide resolved
Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM, you could modify the int to int32_t or int64_t

src/kvstore/NebulaStore.cpp Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

Merging #4101 (2f1ea74) into master (372f6b4) will increase coverage by 0.04%.
The diff coverage is 85.90%.

@@            Coverage Diff             @@
##           master    #4101      +/-   ##
==========================================
+ Coverage   85.03%   85.07%   +0.04%     
==========================================
  Files        1328     1324       -4     
  Lines      131649   131753     +104     
==========================================
+ Hits       111950   112092     +142     
+ Misses      19699    19661      -38     
Impacted Files Coverage Δ
src/common/utils/Types.h 100.00% <ø> (ø)
src/graph/planner/match/MatchClausePlanner.h 100.00% <ø> (ø)
src/kvstore/KVEngine.h 100.00% <ø> (ø)
src/kvstore/RocksEngine.h 83.09% <ø> (-0.54%) ⬇️
src/kvstore/NebulaStore.cpp 67.49% <69.04%> (-3.09%) ⬇️
src/common/utils/NebulaKeyUtils.h 80.41% <77.77%> (-0.27%) ⬇️
src/kvstore/Common.h 82.17% <85.29%> (+3.10%) ⬆️
src/kvstore/RocksEngine.cpp 81.95% <90.00%> (+0.78%) ⬆️
src/common/utils/NebulaKeyUtils.cpp 95.56% <100.00%> (+0.73%) ⬆️
src/graph/planner/match/MatchClausePlanner.cpp 98.78% <100.00%> (+0.12%) ⬆️
... and 370 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f08004a...2f1ea74. Read the comment docs.

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Sophie-Xie Sophie-Xie merged commit 99d93f5 into vesoft-inc:master Apr 6, 2022
Sophie-Xie added a commit that referenced this pull request Apr 7, 2022
* add code for part peers backward compatible

* add balance keys

* fix bug: pass tests

* add gflag && ignore

* change int to int64_t

* chage to emplace

Co-authored-by: panda-sheep <59197347+panda-sheep@users.noreply.github.com>
Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>
yixinglu pushed a commit that referenced this pull request Apr 7, 2022
* fix apply outdate membership change (#4107)

* fix apply outdate membership change

* fix transfer leader to '':0 cause crash

Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>

* add code for part peers backward compatible (#4101)

* add code for part peers backward compatible

* add balance keys

* fix bug: pass tests

* add gflag && ignore

* change int to int64_t

* chage to emplace

Co-authored-by: panda-sheep <59197347+panda-sheep@users.noreply.github.com>
Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>

* fixed (#4116)

Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>

* move KW_CLEAR to unresolved keyword (#4118)

* Limits for add and recover balance data or zone balance jobs (#4104)

* If there are failed or stopped data balance or zone balance job, must firstly recover it

* address wenhaocs's comments

* address comments

* add more comments

* recover job

* add more ut

* adjust comment format

* adjust comment format

* adjust comment format

Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com>
Co-authored-by: pengwei.song <90180021+pengweisong@users.noreply.github.com>
Co-authored-by: panda-sheep <59197347+panda-sheep@users.noreply.github.com>
Co-authored-by: haifei.zhao <32253291+zhaohaifei@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick-v3.1 PR: need cherry-pick to this version ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants