feat!: bootstrap comet cmd for local state sync #16061

tac0turtle · 2023-05-08T11:51:22Z

Description

this pr adds a command to bootstrap the needed functionality for a state sync but without starting the node becase the snapshot is already stored locally

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

confirmed the correct type prefix in the PR title
confirmed ! in the type prefix if API or client breaking change
confirmed all author checklist items have been addressed
reviewed state machine logic
reviewed API design and naming
reviewed documentation is accurate
reviewed tests and test coverage
manually tested (if applicable)

tac0turtle · 2023-05-08T14:10:18Z

this pr is combined with #16060 to complete this feature

yihuang

Not sure if we need to add the basic flags explicitly, the root cmd is different

tac0turtle · 2023-05-08T15:12:56Z

Not sure if we need to add the basic flags explicitly, the root cmd is different

it doesnt look like its needed

mark-rushakoff

This looks mostly fine to me, but use of the height positional CLI argument is my main question.

server/cmt_cmds.go

docs/docs/run-node/01-run-node.md

alexanderbez

utACK

server/cmt_cmds.go

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

server/cmt_cmds.go

docs/docs/run-node/01-run-node.md

server/cmt_cmds.go

julienrbrt · 2023-05-09T20:09:02Z

server/cmt_cmds.go

+	cmd := &cobra.Command{
+		Use:   "bootstrap-state",
+		Short: "Bootstrap CometBFT state at an arbitrary block height using a light client",
+		Long: `


If it is going to say the same thing, why add a long description? I think we should remove it.

server/cmt_cmds.go

julienrbrt · 2023-05-09T20:13:06Z

docs/docs/run-node/01-run-node.md

+
+1. As mentioned in https://docs.cometbft.com/v0.37/core/state-sync, one must set a height and hash in the config.toml along with a few rpc servers (the afromentioned link has instructions on how to do this). 
+2. Bootsrapping Comet state in order to start the node after the snapshot has been ingested. This can be done with the bootstrap command `<app> comet bootstrap-state`
+<!-- 3. TODO after https://github.com/cosmos/cosmos-sdk/pull/16060 is merged -->


Is this blocked on #16060 then?

the cmd behavior itself is standalone and well-defined, but for end user, it only make sense to work together with local snapshot import feature, I think can be merged separately.

server/cmt_cmds.go

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com> (cherry picked from commit 25df90d) # Conflicts: # CHANGELOG.md # server/cmt_cmds.go # server/util.go

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com> (cherry picked from commit 25df90d) # Conflicts: # CHANGELOG.md # docs/run-node/run-node.md # server/cmt_cmds.go # server/util.go

…6080) Co-authored-by: Marko <marbar3778@yahoo.com>

…6079) Co-authored-by: Marko <marbar3778@yahoo.com>

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com>

* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065) Co-authored-by: yihuang <huang@crypto.com> * chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115) Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * refactor!: remove store v2alpha1 (cosmos#15152) * build: use cometbft in v0.46 (cosmos#15090) * chore: bump cometbft to v0.34.27 final (cosmos#15206) * fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245) Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: revert double close fix (cosmos#15255) * chore: prepare v0.46.11 release (cosmos#15259) * feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321) * feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349) * fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383) Co-authored-by: Julien Robert <julien@rbrt.fr> * docs: update cosmwasm link (cosmos#15441) * fix: remove extra `;` in service.proto (cosmos#15443) * fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461) Co-authored-by: Robert Zaremba <robert@zaremba.ch> * feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473) Co-authored-by: Marko <marbar3778@yahoo.com> * docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476) Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525) Co-authored-by: Robert Zaremba <robert@zaremba.ch> * fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667) Co-authored-by: Daniel Wedul <github@wedul.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669) Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * chore: prepare v0.46.12 release notes (cosmos#15685) * fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717) Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775) Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925) Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * build(deps): bump cometbft to v0.34.28 (cosmos#15973) * chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981) Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * feat!: bootstrap comet cmd for local state sync (backport cosmos#16061) (cosmos#16080) Co-authored-by: Marko <marbar3778@yahoo.com> * chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * chore: update sidebar v0.46 (cosmos#16304) * chore: add baseapp circuit breaker setter (cosmos#16289) Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306) Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331) * chore: prepare v0.46.13 (cosmos#16249) * chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409) Co-authored-by: Julien Robert <julien@rbrt.fr> * Add the quarantine and sanction stores to the import-export sim test. * refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432) Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Facundo Medica <facundomedica@gmail.com> * chore: prepare v0.46.13 (2/2) (cosmos#16443) * fix: patch barberry (cosmos#16466) * Better gov prop error handling. * Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do. * Add changelog entries. * Regen the swagger docs. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Robert Zaremba <robert@zaremba.ch> Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: Facundo Medica <facundomedica@gmail.com>

* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065) * chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115) * refactor!: remove store v2alpha1 (cosmos#15152) * build: use cometbft in v0.46 (cosmos#15090) * chore: bump cometbft to v0.34.27 final (cosmos#15206) * fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245) * fix: revert double close fix (cosmos#15255) * chore: prepare v0.46.11 release (cosmos#15259) * feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321) * feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349) * fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383) * docs: update cosmwasm link (cosmos#15441) * fix: remove extra `;` in service.proto (cosmos#15443) * fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461) * feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473) * docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481) * feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476) * fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525) * fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667) * fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669) * chore: prepare v0.46.12 release notes (cosmos#15685) * fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717) * feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775) * feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925) * build(deps): bump cometbft to v0.34.28 (cosmos#15973) * chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981) * feat!: bootstrap comet cmd for local state sync (backport cosmos#16061) (cosmos#16080) * chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087) * feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103) * fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140) * feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205) * feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262) * chore: update sidebar v0.46 (cosmos#16304) * chore: add baseapp circuit breaker setter (cosmos#16289) * fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314) * feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306) * fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331) * chore: prepare v0.46.13 (cosmos#16249) * chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409) * Add the quarantine and sanction stores to the import-export sim test. * refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432) * chore: prepare v0.46.13 (2/2) (cosmos#16443) * fix: patch barberry (cosmos#16466) * Better gov prop error handling. * Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do. * Add changelog entries. * Regen the swagger docs. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Robert Zaremba <robert@zaremba.ch> Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: Facundo Medica <facundomedica@gmail.com>

… (cosmos#16079) Co-authored-by: Marko <marbar3778@yahoo.com>

tac0turtle added 3 commits May 8, 2023 13:50

bootstrap comet for local state sync

74d49c2

change to server.ctx

95c1959

add shrot docs

ff3d06b

tac0turtle force-pushed the marko/bootstrap_cmt branch from 42b2009 to ff3d06b Compare May 8, 2023 13:45

Merge branch 'main' into marko/bootstrap_cmt

4dfdb81

tac0turtle marked this pull request as ready for review May 8, 2023 13:46

tac0turtle requested a review from a team as a code owner May 8, 2023 13:46

tac0turtle requested a review from yihuang May 8, 2023 13:46

yihuang reviewed May 8, 2023

View reviewed changes

mark-rushakoff reviewed May 8, 2023

View reviewed changes

ammend comments

6734f27

alexanderbez reviewed May 8, 2023

View reviewed changes

server/cmt_cmds.go Outdated Show resolved Hide resolved

server/cmt_cmds.go Show resolved Hide resolved

tac0turtle and others added 2 commits May 8, 2023 22:30

Update server/cmt_cmds.go

9f2e35b

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

Update server/cmt_cmds.go

825c84e

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

yihuang reviewed May 9, 2023

View reviewed changes

server/cmt_cmds.go Outdated Show resolved Hide resolved

yihuang added 2 commits May 9, 2023 14:54

bootstrap block store state

261bf90

bootstrap to state height by default

4f058f5

yihuang reviewed May 9, 2023

View reviewed changes

docs/docs/run-node/01-run-node.md Outdated Show resolved Hide resolved

yihuang and others added 2 commits May 9, 2023 15:50

Merge branch 'main' into marko/bootstrap_cmt

143b1a5

minor cleanup

cf45554

yihuang reviewed May 9, 2023

View reviewed changes

server/cmt_cmds.go Outdated Show resolved Hide resolved

yihuang reviewed May 9, 2023

View reviewed changes

server/cmt_cmds.go Outdated Show resolved Hide resolved

Apply suggestions from code review

fe98575

yihuang mentioned this pull request May 9, 2023

feat: add bootstrap-state cmd cometbft/cometbft#803

Closed

3 tasks

yihuang approved these changes May 9, 2023

View reviewed changes

yihuang added backport/0.46.x PR scheduled for inclusion in the v0.46's next stable release backport/v0.47.x PR scheduled for inclusion in the v0.47's next stable release labels May 9, 2023

julienrbrt reviewed May 9, 2023

View reviewed changes

yihuang reviewed May 10, 2023

View reviewed changes

server/cmt_cmds.go Outdated Show resolved Hide resolved

yihuang and others added 2 commits May 10, 2023 10:08

Update server/cmt_cmds.go

5b4b4c6

fix linting

a4e2be9

tac0turtle enabled auto-merge May 10, 2023 09:36

tac0turtle added this pull request to the merge queue May 10, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 10, 2023

julienrbrt added this pull request to the merge queue May 10, 2023

julienrbrt approved these changes May 10, 2023

View reviewed changes

github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 10, 2023

tac0turtle added this pull request to the merge queue May 10, 2023

Merged via the queue into main with commit 25df90d May 10, 2023

tac0turtle deleted the marko/bootstrap_cmt branch May 10, 2023 10:27

This was referenced May 10, 2023

feat!: bootstrap comet cmd for local state sync (backport #16061) #16079

Merged

feat!: bootstrap comet cmd for local state sync (backport #16061) #16080

Merged

tac0turtle added a commit that referenced this pull request May 10, 2023

feat!: bootstrap comet cmd for local state sync (backport #16061) (#1…

738220a

…6080) Co-authored-by: Marko <marbar3778@yahoo.com>

tac0turtle added a commit that referenced this pull request May 11, 2023

feat!: bootstrap comet cmd for local state sync (backport #16061) (#1…

050f03a

…6079) Co-authored-by: Marko <marbar3778@yahoo.com>

kocubinski pushed a commit that referenced this pull request May 11, 2023

feat!: bootstrap comet cmd for local state sync (#16061)

f3272d2

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: HuangYi <huang@crypto.com>

yihuang mentioned this pull request May 22, 2023

Enable local snapshot restore (backport: #733) cometbft/cometbft#728

Closed

3 tasks

cason mentioned this pull request May 23, 2023

State sync from local snapshot: Understand problem, explore solutions cometbft/cometbft#28

Closed

4 tasks

faddat mentioned this pull request Jun 8, 2023

barberry realiotech/cosmos-sdk#30

Merged

roy-dydx pushed a commit to dydxprotocol/cosmos-sdk that referenced this pull request Jul 11, 2023

feat!: bootstrap comet cmd for local state sync (backport cosmos#16061)…

f90f25c

… (cosmos#16079) Co-authored-by: Marko <marbar3778@yahoo.com>

LampardNguyen234 mentioned this pull request Aug 15, 2023

deps: bump cosmos-sdk v0.46 and evmos v12 AstraProtocol/astra#209

Merged

kakysha pushed a commit to InjectiveLabs/cosmos-sdk that referenced this pull request Aug 15, 2023

feat!: bootstrap comet cmd for local state sync (backport cosmos#16061)…

99b3714

… (cosmos#16079) Co-authored-by: Marko <marbar3778@yahoo.com>

0xstepit mentioned this pull request Jul 1, 2024

chore(cachemulti): add Copy method evmos/cosmos-sdk#52

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: bootstrap comet cmd for local state sync #16061

feat!: bootstrap comet cmd for local state sync #16061

tac0turtle commented May 8, 2023

tac0turtle commented May 8, 2023

yihuang left a comment

tac0turtle commented May 8, 2023

mark-rushakoff left a comment

alexanderbez left a comment

julienrbrt May 9, 2023

julienrbrt May 9, 2023

yihuang May 10, 2023 •

edited

Loading

feat!: bootstrap comet cmd for local state sync #16061

feat!: bootstrap comet cmd for local state sync #16061

Conversation

tac0turtle commented May 8, 2023

Description

Author Checklist

Reviewers Checklist

tac0turtle commented May 8, 2023

yihuang left a comment

Choose a reason for hiding this comment

tac0turtle commented May 8, 2023

mark-rushakoff left a comment

Choose a reason for hiding this comment

alexanderbez left a comment

Choose a reason for hiding this comment

julienrbrt May 9, 2023

Choose a reason for hiding this comment

julienrbrt May 9, 2023

Choose a reason for hiding this comment

yihuang May 10, 2023 • edited Loading

Choose a reason for hiding this comment

yihuang May 10, 2023 •

edited

Loading