Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem: there's no compact historical state storage #704

Closed
yihuang opened this issue Sep 19, 2022 · 2 comments
Closed

Problem: there's no compact historical state storage #704

yihuang opened this issue Sep 19, 2022 · 2 comments

Comments

@yihuang
Copy link
Collaborator

yihuang commented Sep 19, 2022 •
edited
Loading

A solution for: #451
Since we already have the state streaming in SDK(ADR 038), it's possible to implement the idea in Cronos itself.
Just implement a streaming service that store the state changes in a kv db, the format is explained here.
@yihuang
Copy link
Collaborator Author

yihuang commented Sep 27, 2022 •
edited
Loading

File streamer integration

yihuang added a commit to yihuang/cronos that referenced this issue Sep 27, 2022
@yihuang
Problem: there's no compact historical state storage
cc3ac31 
Closes: crypto-org-chain#704
Solution:
- Integration version store and streaming service.
@yihuang yihuang mentioned this issue Sep 27, 2022
Closed
13 tasks
yihuang added a commit to yihuang/cronos that referenced this issue Sep 27, 2022
@yihuang
Problem: there's no compact historical state storage
760f02a 
Closes: crypto-org-chain#704
Solution:
- Integration version store and streaming service.
yihuang added a commit to yihuang/cronos that referenced this issue Sep 27, 2022
@yihuang
Problem: there's no compact historical state storage
f6816f2 
Closes: crypto-org-chain#704
Solution:
- Integration version store and streaming service.
@yihuang yihuang mentioned this issue Sep 28, 2022
Open
1 task
yihuang added a commit to yihuang/cronos that referenced this issue Oct 7, 2022
@yihuang
Problem: there's no compact historical state storage
65035a9 
Closes: crypto-org-chain#704
Solution:
- Integration version store and streaming service.

multiple backends

db size benchmark

support mdbx backend through tm-db

store latest version

support GetLatestVersion

query multistore

test versiondb streamer
yihuang added a commit to yihuang/cronos that referenced this issue Nov 28, 2022
@yihuang
Problem: there's no compact historical state storage
3bcb198 
Closes: crypto-org-chain#704
Solution:
- Integration version store and streaming service.

multiple backends

db size benchmark

support mdbx backend through tm-db

store latest version

support GetLatestVersion

query multistore

test versiondb streamer

fix lint

fix state listener

temp

fix state listening

optimize common case

fix lint

try to fix mdbx build

update cosmos-sdk

fix clone append

add test case

fix subkey problem in history index

revert chunking, hard to work with variable length key

support iterator

check future height

fix lint

new state listener

fix latest state in query

fix integration test

fix prune node test

update dependency

add utility to read from file streamer

Update versiondb/multistore.go

Signed-off-by: yihuang <huang@crypto.com>

add unit test

create common backend test cases

update dependency

update with new file streamer format

Problem: python3.10 is not used in integration tests

Solution:
- start using python3.10, prepare to later PRs which need the new features
- update nixpkgs nesserary for the nix stuff to work.

python-roaring64

remove debug log

add test cases, improve coverage
@yihuang yihuang mentioned this issue Jan 19, 2023
Merged
13 tasks
@yihuang
Copy link
Collaborator Author

yihuang commented Feb 13, 2023

closed by #791

@yihuang yihuang closed this as completed Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Problem: there's no compact historical state storage yihuang/cronos
1 participant
@yihuang