Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[resharding] Nightshade V3 shard layout for protocol version 65 #10725

Merged
merged 7 commits into from
Mar 11, 2024

Conversation

shreyan-gupta
Copy link
Contributor

@shreyan-gupta shreyan-gupta commented Mar 7, 2024

This PR is to be cherry-picked into release 1.38.0 for splitting shard 2 into two parts.

Hot and kai-ching both fall on shard 2 which has been causing a lot of congestion.

Zulip thread: https://near.zulipchat.com/#narrow/stream/308695-nearone.2Fprivate/topic/constant.20congestion.20on.20shard.202/near/425367222

@@ -184,6 +185,7 @@ impl ProtocolFeature {
ProtocolFeature::RestrictTla
| ProtocolFeature::TestnetFewerBlockProducers
| ProtocolFeature::SimpleNightshadeV2 => 64,
ProtocolFeature::SimpleNightshadeV3 => 65,
Copy link
Contributor Author

@shreyan-gupta shreyan-gupta Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be noted, STABLE_PROTOCOL_VERSION is already set as 65 on master. We may need to change this while cherry picking to 1.38.0

Copy link

codecov bot commented Mar 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.70%. Comparing base (d50e30c) to head (b0be300).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10725      +/-   ##
==========================================
+ Coverage   71.66%   71.70%   +0.03%     
==========================================
  Files         756      756              
  Lines      152299   152362      +63     
  Branches   152299   152362      +63     
==========================================
+ Hits       109150   109246      +96     
+ Misses      38186    38139      -47     
- Partials     4963     4977      +14     
Flag Coverage Δ
backward-compatibility 0.24% <0.00%> (-0.01%) ⬇️
db-migration 0.24% <0.00%> (-0.01%) ⬇️
genesis-check 1.42% <0.00%> (-0.01%) ⬇️
integration-tests 36.86% <34.92%> (+0.09%) ⬆️
linux 70.49% <100.00%> (+<0.01%) ⬆️
linux-nightly 71.15% <100.00%> (+0.03%) ⬆️
macos 54.84% <96.82%> (+0.26%) ⬆️
pytests 1.65% <0.00%> (-0.01%) ⬇️
sanity-checks 1.43% <0.00%> (-0.01%) ⬇️
unittests 67.57% <96.82%> (+0.01%) ⬆️
upgradability 0.29% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wacban
Copy link
Contributor

wacban commented Mar 7, 2024

@pugachAG Are we going to need that cache/ prefetcher overrides as for shard 3?

@pugachAG
Copy link
Contributor

pugachAG commented Mar 7, 2024

@wacban yes, you need to update per_shard_max_bytes config for the new shard layout:
https://github.com/near/nearcore/blob/8e0bdf35153bb830521788451a57585ca6263573/core/store/src/config.rs#L229C17-L242

should be pretty straightforward

@@ -235,10 +235,13 @@ impl Default for StoreConfig {
(ShardUId { version: 1, shard_id: 3 }, bytesize::ByteSize::gb(3)),
// In simple nightshade v2 the heavy contract "token.sweat" is in shard 4
(ShardUId { version: 2, shard_id: 4 }, bytesize::ByteSize::gb(3)),
// In simple nightshade v3 the heavy contract "token.sweat" is in shard 5
(ShardUId { version: 3, shard_id: 5 }, bytesize::ByteSize::gb(3)),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming, shard_id gets pushed by one here

Copy link
Contributor

@wacban wacban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wacban wacban added this pull request to the merge queue Mar 11, 2024
Merged via the queue into near:master with commit 0efb74b Mar 11, 2024
23 of 24 checks passed
@wacban
Copy link
Contributor

wacban commented Mar 11, 2024

cc @posvyatokum and @marcelo-gonzalez I merged the new shard layout in preparation for 1.38 release. I will merge the integration tests for it next.

marcelo-gonzalez pushed a commit to marcelo-gonzalez/nearcore that referenced this pull request Mar 11, 2024
…#10725)

This PR is to be cherry-picked into release 1.38.0 for splitting shard 2
into two parts.

Hot and kai-ching both fall on shard 2 which has been causing a lot of
congestion.

Zulip thread:
https://near.zulipchat.com/#narrow/stream/308695-nearone.2Fprivate/topic/constant.20congestion.20on.20shard.202/near/425367222

---------

Co-authored-by: wacban <wac.banasik@gmail.com>
github-merge-queue bot pushed a commit that referenced this pull request Mar 11, 2024
Please only refer to the last commit
`7b289c9e763e96b6c863baeb7b80f9ec9c638620` or changes in file
`integration-tests/src/tests/client/resharding.rs`

I'm unable to create stacked PRs for my forked repo.

To be merged only after #10725

---------

Co-authored-by: wacban <wac.banasik@gmail.com>
marcelo-gonzalez pushed a commit to marcelo-gonzalez/nearcore that referenced this pull request Mar 11, 2024
…#10725)

This PR is to be cherry-picked into release 1.38.0 for splitting shard 2
into two parts.

Hot and kai-ching both fall on shard 2 which has been causing a lot of
congestion.

Zulip thread:
https://near.zulipchat.com/#narrow/stream/308695-nearone.2Fprivate/topic/constant.20congestion.20on.20shard.202/near/425367222

---------

Co-authored-by: wacban <wac.banasik@gmail.com>
@shreyan-gupta shreyan-gupta deleted the shreyan/resharding/v3_shard_layout branch March 12, 2024 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants