Fast Cache - Downgrade - reupgrade protection and other improvements #12

p0mvn · 2022-01-26T23:58:56Z

#11 - Background

We would like to add downgrade and re-upgrade protection for the fast cache. Write the last fast node version on each block and check it at the startup. If not set to the most recent version, rewrite fast cache

Acceptance criteria

protection added
unit tested

Additionally:

optimized non-membership proof
unit tested fast node
resolved a bug related to the location of where migration was applied in LoadVersion and LazyLoadVersion
looked into optimizing GetWithIdndex and GetByIndex
- the only idea made it worse off so had to revert
bench tests for GetWithIndex, GetByIndex and `

Latest benchmark for non-membership proof:

Running tool: /usr/local/go/bin/go test -benchmem -run=^$ -bench ^BenchmarkGetNonMembership$ github.com/cosmos/iavl

goos: linux
goarch: amd64
pkg: github.com/cosmos/iavl
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
BenchmarkGetNonMembership/fast-8         	    1130	   1003260 ns/op	  132431 B/op	    2920 allocs/op
BenchmarkGetNonMembership/regular-8      	     154	   7399486 ns/op	 1292050 B/op	   21474 allocs/op
PASS
ok  	github.com/cosmos/iavl	54.441s

Old benchmark from dev/iavl_data_locality with modified bench tests:

iavl: 
git commit: 
git branch: 
go version go1.17.6 linux/amd64

Init Tree took 78.76 MB
goos: linux
goarch: amd64
pkg: github.com/cosmos/iavl/benchmarks
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
BenchmarkMedium/memdb-100000-100-16-40/query-no-in-tree-guarantee-slow-8         	   93859	     12319 ns/op	     592 B/op	      12 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/query-hits-slow-8                         	   83343	     14889 ns/op	     760 B/op	      15 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/iteration-slow-8                          	       2	 829462950 ns/op	88838692 B/op	 1600081 allocs/op
--- BENCH: BenchmarkMedium/memdb-100000-100-16-40/iteration-slow-8
    bench_test.go:128: completed 100000 iterations
    bench_test.go:128: completed 100000 iterations
    bench_test.go:128: completed 100000 iterations
BenchmarkMedium/memdb-100000-100-16-40/update-8                                  	   10000	    149613 ns/op	   27337 B/op	     335 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/block-8                                   	      78	  16881365 ns/op	 2910816 B/op	   35693 allocs/op
Init Tree took 40.64 MB
BenchmarkMedium/goleveldb-100000-100-16-40/query-no-in-tree-guarantee-slow-8     	   56197	     21425 ns/op	    1566 B/op	      30 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/query-hits-slow-8                     	   43808	     27835 ns/op	    2122 B/op	      39 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/iteration-slow-8                      	       1	1974250600 ns/op	226939520 B/op	 3862227 allocs/op
--- BENCH: BenchmarkMedium/goleveldb-100000-100-16-40/iteration-slow-8
    bench_test.go:128: completed 100000 iterations
BenchmarkMedium/goleveldb-100000-100-16-40/update-8                              	    8377	    211953 ns/op	   43961 B/op	     425 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/block-8                               	      45	  28982847 ns/op	 5162017 B/op	   53449 allocs/op
PASS
ok  	github.com/cosmos/iavl/benchmarks	25.442s

Latest benchmark:

Running tool: /usr/local/go/bin/go test -benchmem -run=^$ -bench ^BenchmarkMedium$ github.com/cosmos/iavl/benchmarks

iavl: 
git commit: 
git branch: 
go version go1.17.6 linux/amd64

Init Tree took 114.29 MB
goos: linux
goarch: amd64
pkg: github.com/cosmos/iavl/benchmarks
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
BenchmarkMedium/memdb-100000-100-16-40/query-no-in-tree-guarantee-fast-8         	  596952	      2612 ns/op	     112 B/op	       4 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/query-no-in-tree-guarantee-slow-8         	   96513	     13278 ns/op	     440 B/op	       8 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/query-hits-fast-8                         	  737870	      1686 ns/op	      26 B/op	       0 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/query-hits-slow-8                         	   69254	     19333 ns/op	     605 B/op	      11 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/iteration-fast-8                          	      16	 101840581 ns/op	18400314 B/op	  300000 allocs/op
--- BENCH: BenchmarkMedium/memdb-100000-100-16-40/iteration-fast-8
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
	... [output truncated]
BenchmarkMedium/memdb-100000-100-16-40/iteration-slow-8                          	       2	1075280050 ns/op	88843124 B/op	 1600102 allocs/op
--- BENCH: BenchmarkMedium/memdb-100000-100-16-40/iteration-slow-8
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
BenchmarkMedium/memdb-100000-100-16-40/update-8                                  	    8095	    226336 ns/op	   27376 B/op	     341 allocs/op
BenchmarkMedium/memdb-100000-100-16-40/block-8                                   	      52	  22879244 ns/op	 2946879 B/op	   36431 allocs/op
Init Tree took 66.82 MB
BenchmarkMedium/goleveldb-100000-100-16-40/query-no-in-tree-guarantee-fast-8     	  238384	      6958 ns/op	     814 B/op	      16 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/query-no-in-tree-guarantee-slow-8     	   33271	     34922 ns/op	    1844 B/op	      32 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/query-hits-fast-8                     	  459201	      2207 ns/op	     114 B/op	       2 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/query-hits-slow-8                     	   38526	     32942 ns/op	    1987 B/op	      33 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/iteration-fast-8                      	      10	 109896370 ns/op	29327281 B/op	  522988 allocs/op
--- BENCH: BenchmarkMedium/goleveldb-100000-100-16-40/iteration-fast-8
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
    bench_test.go:143: completed 100000 iterations
	... [output truncated]
BenchmarkMedium/goleveldb-100000-100-16-40/iteration-slow-8                      	       1	2197231400 ns/op	236026680 B/op	 3998799 allocs/op
--- BENCH: BenchmarkMedium/goleveldb-100000-100-16-40/iteration-slow-8
    bench_test.go:143: completed 100000 iterations
BenchmarkMedium/goleveldb-100000-100-16-40/update-8                              	    7573	    281499 ns/op	   43491 B/op	     432 allocs/op
BenchmarkMedium/goleveldb-100000-100-16-40/block-8                               	      30	  49443073 ns/op	 6370082 B/op	   66312 allocs/op
PASS
ok  	github.com/cosmos/iavl/benchmarks	49.589s

…on and lazy load version

p0mvn · 2022-01-29T01:13:58Z

It seems that GetwithIndex and GetByIndex have actually become worse off:

// OLD Implementation

Benchmark_GetWithIndex/regular-8      	   78734	     17354 ns/op	     720 B/op	      15 allocs/op

Benchmark_GetByIndex/regular-8      	   71974	     17146 ns/op	     822 B/op	      15 allocs/op


// NEW Implementation

Benchmark_GetWithIndex/fast-8         	      44	  23212700 ns/op	 3865367 B/op	   96616 allocs/op
Benchmark_GetWithIndex/regular-8      	   85010	     14220 ns/op	     584 B/op	      11 allocs/op

Benchmark_GetByIndex/fast-8         	     100	  20810321 ns/op	 3821985 B/op	   95532 allocs/op
Benchmark_GetByIndex/regular-8      	   72451	     17056 ns/op	     828 B/op	      15 allocs/op

Using fast node iterations to GetWithIndex and GetByIndex might not be the most optimal solution. It does make sense because traversing the tree allows searching for the correct key in O(log(n)) whereas iterations in O(n). The fact that O(n) fast iteration preserves data locality does not help to outperform O(log(n)) tree search with no data locality

I reverted the changes for GetWithIndex and GetByIndex.

The non-membership proof is still better off, however. Even as I was increasing the key size, the difference between old and new implementations was growing.

p0mvn · 2022-01-29T05:12:30Z

@ValarDragon Please review. This branch is diffed against my older PR #5. It also needs some more 👀

ValarDragon · 2022-01-30T00:07:04Z

mutable_tree.go

@@ -760,10 +761,8 @@ func (tree *MutableTree) saveFastNodeVersion() error {
 		return err
 	}

-	if !tree.ndb.isFastStorageEnabled() {


This is removed because its only called if tree.ndb.isFastStorageEnabled is true?

If so, can we add a comment stating this above the saveFastNodeVersion function?

The reason why this was removed is that now we are going with the assumption that fast cache storage will be upgraded to on all versions above 1.1.0 or fastStorageVersionValue. Therefore, we must make sure that we persist the latest live version on disk. Otherwise, re-upgrade protection would not function correctly.

In case there is any confusion, there are now 2 separate methods that check for fast storage to be enabled:

in nodeDB (hasUpgradedToFastStorage)

This method checks that the upgrade has happened. When the upgrade is successful, fast storage on disk must match the latest tree.

I renamed this method from isFastStorageEnabled to hasUpgradedToFastStorage and added a comment for clarity

in immutable tree (isFastStorageEnabled)

This one checks for 2 conditions:

The tree is of the latest version

hasUpgradedToFastStorage - this one is more of a sanity check since now we are assuming that the upgrade should persist forever unless manually downgraded.

also added a comment

Let me know if this makes sense

ValarDragon · 2022-01-30T00:15:12Z

proof_ics23.go

+	var nextKey []byte = nil
+
+	done := false
+	itr := t.Iterator(nil, nil, true)


Hrmm, isn't iterating over all of fast node state going to be slower than a logarithmic number of file opens? Is there a way we can set a 'smarter' start bound?

I think we can do this speedup in a non-db breaking way later, so may also be worth commenting out its invocation in GetNonMembershipProof and revisiting after we get the initial release out!

tree_test.go

ValarDragon · 2022-01-30T00:31:05Z

nodedb.go

+			return true
+		}
+	}
+	return false


This should never happen under correct behavior right? Should we add a comment do a fmt.Println if this occurs?

This should happen when we have already upgraded to fast storage and it matches the latest state to avoid redundantly upgrading fast nodes when they are already live

This should also happen if we are on the default (non-fast) storage version. This method is independent from hasUpgradedToFastStorage.

To avoid confusion,

hasUpgradedToFastStorage - returns true if we have upgraded to fast storage, false if we are on default

shouldForceFastStorageUpdate - returns true if we have upgraded to fast storage but it does not match the live state, false otherwise

nodedb.go

ValarDragon · 2022-01-30T07:12:06Z

nodedb.go

@@ -211,6 +219,10 @@ func (ndb *nodeDB) SaveFastNode(node *FastNode) error {
 }

 func (ndb *nodeDB) setStorageVersion(newVersion string) error {
+	if newVersion >= fastStorageVersionValue {
+		newVersion = newVersion + fastStorageVersionDelimiter + strconv.Itoa(int(ndb.latestVersion))


won't we be infinitely appending to newVersion like this, if newVersion gets parsed from the serialized value later?

I don't think any of the tests test repeated calls of parsing & setting but maybe i'm wrong

You are right, good catch.

I refactored this method completely and renamed it to setFastStorageVersionToBatch. Now, it does not take version as a parameter. Parses the storageVersion and updates it to fast, if needed, with the latest live tree version. Added more unit tests to make sure all code branches are covered

…it test

… comments

…edToFastStorage and shouldForceFastStorageUpdate

p0mvn · 2022-01-31T20:58:51Z

@ValarDragon addressed all comments so far. The most recent updates:

commented out GetNonMembershipProof new implementation for now
addressed the bug about appending version and unit tested
added more unit tests in mutable_tree_test.go for re-upgrade protection
renamed some methods for clarity and added comments

Please take a look when you have time

…t support 0.35.0

…ed to be loaded

ValarDragon · 2022-02-07T16:02:43Z

Changes LGTM!!

…12) * add leaf hash to fast node and unit test * refactor get with index and get by index, fix migration in load version and lazy load version * use Get in GetVersioned of mutable tree * refactor non membership proof to use fast storage if available * bench non-membership proof * fix bench tests to work with the new changes * add downgrade-reupgrade protection and unit test * remove leaf hash from fast node * resolve multithreading bug related to iterators not being closed * clean up * use correct tree in bench tests * add cache to tree used to bench non membership proofs * add benc tests for GetWithIndex and GetByIndex * revert GetWithIndex and GetByIndex * remove unused import * unit test re-upgrade protection and fix small issues * remove redundant setStorageVersion method * fix bug with appending to live stage version to storage version and nit test * add comment for setFastStorageVersionToBatch * refactor and improve unit tests for reupgrade protection * rename ndb's isFastStorageEnabled to hasUpgradedToFastStorage and add comments * comment out new implementation for GetNonMembershipProof * update comments in nodedb to reflect the difference between hasUpgradedToFastStorage and shouldForceFastStorageUpdate * refactor nodedb tests * downgrade tendermint to 0.34.14 - osmosis's latest cosmos sdk does not support 0.35.0 * fix bug where fast storage was not enabled when version 0 was attempted to be loaded * implement unsaved fast iterator to be used in mutable tree (#16)

* Add unbounded key string to KeyFormat * Add test vectors for unbounded length keys * Add some notes * update .gitignore * Add FastNode struct * WIP: make Get work with new FastNode * when retrieving fastnode fails, return errors vs. panic * add comments clarifying what index represents * make the linter happy * Add small tweaks to fast_node.go * add TODO & small linter tweaks * Update comment * update fast node cache in set * add debugging output when falling back to original logic * return error instead of panic * WIP: refactor set ops to work with fast store * update Set of mutable tree, begin unit testing * update GetVersioned to check fast nodes before trying the immutable * check fast node version before nil value check in get of immutable tree * fix small bugs and typos, continue writing unit tests for Set * unit test saveFastNodeVersion * simplify storing unsaved fast nodes * resolve a bug with not writing prefix for fast node to disk * remove fast nodes from disk on save and clear fast cache when version is deleted, fix all tests but random and with index * resolve an issue with randomized tests caused by the fast node cache not being cleared when latest version is saved * split unsaved fast node changes into additions and removals * save fast node removals * move fast node cache clearing to nodedb * use regular logic only when fast node version is greater than immutable tree' * clean up tree_random_test.go * clear unsaved fast node removals on rollback * fix randomized test failures caused by a typo in ndb DeleteVersion for loop * implement GetFast method to preserve Get with correct index return, convert unit tests from Get to GetFast where applicable * ensure Get and GetFast return the same values in tree_random_test.go * test fast node cache is live in random tree tests * improve mutable tree unit tests related to new functionality * clean up tree_test.go * implement GetVersionedFast to preserve the index in GetVersioned * restore accidentally deleted test in mutable tree test * spurious whitespace * refactor mutable tree * fix comment in mutable tree * add benchmark results * avoid redundant full tree search in GetFast of immutable tree when fast node is nil and tree is latest * fix naming for bench test get on random keys * use method for get latestversio in get fast * optimize GetFast, perform a refactor to ensure that fast nodes on disk are matched and better test * add latest bench * Fast Node Iteration (osmosis-labs#7) * propagate errors from nodedb and avoid panicking * begin implementing fast node iteration * resolve rebase issue in random tests * fix iteration to deserialize fast node for extracting the correct value * finalzie iteration and let all unit tests pass * add benchmarks * merge GetVersioned and GetVersionedFast * remove fast node deletion from DeleteVersion and DeleteVersionRange and benchmark * fix and unit test iteration on mutable and immutable trees * implement tests for iterator and iterate, begin testing fast iterator * fix and unit test fast iterator * refactor iterate methods of mutable and immutable trees * resolve some warnings * remove old bench results * refactor bench tests for iteration * Fast Cache Migration (osmosis-labs#9) * implement nodedb changes to set and get chain version from the database * implement and unit test upgrade to fast storage in mutable tree * refactor for auto upgrade to fast version in mutable tree contructor, load version and lazy load version * use proper functionality for getting latest version * remove unused error * fix comment * use fast version value in tests * spurious tab * fix style problems and remove redundant code in tests * Rename Get to GetWithIndex and GetFast to Get * refactor and clean up bench tests * remove extra byte from fast node length * clean up immutable tree * refactor nil tree or ndb error for the iterators and their tests * avoid exporting methods for getting unsaved additions and removals * refactor fast upgrade to read from tree instead of traversing disk nodes and orphans, update unit tests * remove unneeded comment * refer to storage version consistently across the project * fix more warnings * optimize removal of fast nodes from cache on deletion * small changes in teh mutable tree * correct storage version key * auto set fast version in SaveVersion * avoid exporting new methods of the immutable tree * go fmt * Fix comment in fast iterator Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * add extra comment for domain in fast iterator Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * add comments for moving the iterator before the first element * add comment for describing what mirror is in assertIterator of testutils * fix checking the mirror for descending iterator in tests * Update testutils_test.go with a comment Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * Update benchmarks/bench_test.go with a comment for runKnownQueriesFast Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * Update benchmarks/bench_test.go with a comment for runQueriesFast Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * export IsFastCacheEnabled and add an assert in bench tests * Update comment immutable_tree.go Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * Update comment for migration in mutable_tree.go Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * simlify Iterate in mutable tree, add debug log for * Fast Cache - Downgrade - reupgrade protection and other improvements (osmosis-labs#12) * add leaf hash to fast node and unit test * refactor get with index and get by index, fix migration in load version and lazy load version * use Get in GetVersioned of mutable tree * refactor non membership proof to use fast storage if available * bench non-membership proof * fix bench tests to work with the new changes * add downgrade-reupgrade protection and unit test * remove leaf hash from fast node * resolve multithreading bug related to iterators not being closed * clean up * use correct tree in bench tests * add cache to tree used to bench non membership proofs * add benc tests for GetWithIndex and GetByIndex * revert GetWithIndex and GetByIndex * remove unused import * unit test re-upgrade protection and fix small issues * remove redundant setStorageVersion method * fix bug with appending to live stage version to storage version and nit test * add comment for setFastStorageVersionToBatch * refactor and improve unit tests for reupgrade protection * rename ndb's isFastStorageEnabled to hasUpgradedToFastStorage and add comments * comment out new implementation for GetNonMembershipProof * update comments in nodedb to reflect the difference between hasUpgradedToFastStorage and shouldForceFastStorageUpdate * refactor nodedb tests * downgrade tendermint to 0.34.14 - osmosis's latest cosmos sdk does not support 0.35.0 * fix bug where fast storage was not enabled when version 0 was attempted to be loaded * implement unsaved fast iterator to be used in mutable tree (osmosis-labs#16) * address comments from unsaved fast iterator PR * expose isUpgradeable method on mutable tree and unit test (osmosis-labs#17) * expose isUpgradeable method on mutable tree and unit test * go fmt * resolve problems with rebasing * update CHANGELOG.md * tendermint v0.35.0 * fix String() bench * fix duplication lint in iterator_test.go * fix lint for tree.ndb.DeleteFastNode error return not checked * fix Error return value of `ndb.resetBatch` is not checked * fix Error return value of `ndb.traversePrefix` is not checked * fix Error: struct of size 64 bytes could be of size 56 bytes * fix Error: `comitted` is a misspelling of `committed` * fix issues in basic_test.go * address comments in fast_iterator.go * address comments in immutable tree * address comments in iterator.go * address comments in key_format.go * address remaining comments * fix Error: Error return value of `ndb.batch.Write` is not checked * fix Error: receiver name t should be consistent with previous receiver name tree for MutableTree * fix Error: struct of size 48 bytes could be of size 40 bytes * go fmt * more linter fixes * fix remaining linter problems * upgrade tm-db and comment out broken bencher databases * skip iterations for BenchmarkLevelDBLargeData bench * attempt to fix linter * force GC, no cache during migration, auto heap profile (osmosis-labs#19) * force GC, no cache during migration, auto heap profile * resolve a potential deadlock from racing between reset and stop * fix small lint issue * remove logs and pprof logic * remove unused libraries * add comment explaining the reason for RAM optimizations * sync access to fast node cache to avoid concurrent write fatal error (osmosis-labs#23) * update go.mod and go.sum to match master versions * revert osmosis-labs#23 (sync access to fast node cache), fix bug related to old height export (osmosis-labs#33) * Revert "sync access to fast node cache to avoid concurrent write fatal error (osmosis-labs#23)" This reverts commit 2a1daf4. * return correct iterator in mutable tree * fix concurrent map panic when querying and comittting concurrently * avoid clearing fast node cache during pruning (osmosis-labs#35) * fix data race related to VersionExists (osmosis-labs#36) * fix data race related to VersionExists * use regular lock instead of RW in mutable_tree.go * hardcode fast node cache size to 100k * go fmt * restore proof_ics23.go * fix linter Co-authored-by: ValarDragon <dojha12@gmail.com> Co-authored-by: jtieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Roman Akhtariev <34196718+akhtariev@users.noreply.github.com> Co-authored-by: Roman <34196718+r0mvn@users.noreply.github.com>

p0mvn added 2 commits January 25, 2022 18:31

add leaf hash to fast node and unit test

6350dc8

refactor get with index and get by index, fix migration in load versi…

5f579c5

…on and lazy load version

This was linked to issues Jan 26, 2022

Fast Cache - Optimize GetWithIndex by including leaf hash to fast nodes #10

Closed

Fast Cache - Downgrade and and re-upgrade protection #11

Closed

p0mvn added 11 commits January 27, 2022 10:19

use Get in GetVersioned of mutable tree

f6ebabb

refactor non membership proof to use fast storage if available

89633f7

bench non-membership proof

75d9f81

fix bench tests to work with the new changes

19ee7d3

add downgrade-reupgrade protection and unit test

cbf2f41

remove leaf hash from fast node

96208dc

resolve multithreading bug related to iterators not being closed

5217e71

clean up

ea072a6

use correct tree in bench tests

806b1a0

add cache to tree used to bench non membership proofs

3f24e71

add benc tests for GetWithIndex and GetByIndex

0cb480e

p0mvn added 2 commits January 28, 2022 20:51

revert GetWithIndex and GetByIndex

ee1ab84

remove unused import

2edb8cc

p0mvn mentioned this pull request Jan 29, 2022

Fast Cache - Optimize GetWithIndex by including leaf hash to fast nodes #10

Closed

5 tasks

p0mvn changed the title ~~Fast Cache - Optimizations for GetByIndex, GetWithIndex and downgrade - reupgrade protection~~ Fast Cache - Downgrade - reupgrade protection and other improvements Jan 29, 2022

p0mvn requested a review from ValarDragon January 29, 2022 05:10

ValarDragon reviewed Jan 30, 2022

View reviewed changes

tree_test.go Show resolved Hide resolved

ValarDragon reviewed Jan 30, 2022

View reviewed changes

nodedb.go Outdated Show resolved Hide resolved

ValarDragon reviewed Jan 30, 2022

View reviewed changes

p0mvn added 2 commits January 30, 2022 09:42

unit test re-upgrade protection and fix small issues

9712e87

remove redundant setStorageVersion method

eea8cfd

p0mvn added 7 commits January 30, 2022 20:27

fix bug with appending to live stage version to storage version and n…

cd0a61c

…it test

add comment for setFastStorageVersionToBatch

8586d10

refactor and improve unit tests for reupgrade protection

c293938

rename ndb's isFastStorageEnabled to hasUpgradedToFastStorage and add…

eae9425

… comments

comment out new implementation for GetNonMembershipProof

2241204

update comments in nodedb to reflect the difference between hasUpgrad…

31710b3

…edToFastStorage and shouldForceFastStorageUpdate

refactor nodedb tests

c0e4776

p0mvn added 2 commits February 3, 2022 12:35

downgrade tendermint to 0.34.14 - osmosis's latest cosmos sdk does no…

51b6c2f

…t support 0.35.0

fix bug where fast storage was not enabled when version 0 was attempt…

a7e7fb2

…ed to be loaded

ValarDragon approved these changes Feb 7, 2022

View reviewed changes

implement unsaved fast iterator to be used in mutable tree (#16)

f78896b

p0mvn merged commit 6e08ce8 into roman/fast-node-get-set Feb 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fast Cache - Downgrade - reupgrade protection and other improvements #12

Fast Cache - Downgrade - reupgrade protection and other improvements #12

p0mvn commented Jan 26, 2022 •

edited

Loading

p0mvn commented Jan 29, 2022 •

edited

Loading

p0mvn commented Jan 29, 2022

ValarDragon Jan 30, 2022 •

edited

Loading

p0mvn Jan 31, 2022

ValarDragon Jan 30, 2022 •

edited

Loading

ValarDragon Jan 30, 2022

ValarDragon Jan 30, 2022

p0mvn Jan 31, 2022

p0mvn Jan 31, 2022

ValarDragon Jan 30, 2022

ValarDragon Jan 30, 2022 •

edited

Loading

p0mvn Jan 31, 2022

p0mvn commented Jan 31, 2022

ValarDragon commented Feb 7, 2022

Fast Cache - Downgrade - reupgrade protection and other improvements #12

Fast Cache - Downgrade - reupgrade protection and other improvements #12

Conversation

p0mvn commented Jan 26, 2022 • edited Loading

p0mvn commented Jan 29, 2022 • edited Loading

p0mvn commented Jan 29, 2022

ValarDragon Jan 30, 2022 • edited Loading

Choose a reason for hiding this comment

p0mvn Jan 31, 2022

Choose a reason for hiding this comment

ValarDragon Jan 30, 2022 • edited Loading

Choose a reason for hiding this comment

ValarDragon Jan 30, 2022

Choose a reason for hiding this comment

ValarDragon Jan 30, 2022

Choose a reason for hiding this comment

p0mvn Jan 31, 2022

Choose a reason for hiding this comment

p0mvn Jan 31, 2022

Choose a reason for hiding this comment

ValarDragon Jan 30, 2022

Choose a reason for hiding this comment

ValarDragon Jan 30, 2022 • edited Loading

Choose a reason for hiding this comment

p0mvn Jan 31, 2022

Choose a reason for hiding this comment

p0mvn commented Jan 31, 2022

ValarDragon commented Feb 7, 2022

p0mvn commented Jan 26, 2022 •

edited

Loading

p0mvn commented Jan 29, 2022 •

edited

Loading

ValarDragon Jan 30, 2022 •

edited

Loading

ValarDragon Jan 30, 2022 •

edited

Loading

ValarDragon Jan 30, 2022 •

edited

Loading