Randomized Fuzzer: tried to delete a non-existent validator

[ValarDragon]

Steps to Reproduce

I got the following from test_sim_modules on develop, in the staking simulation. Re-running a few times will probably recreate another failing seed.

Starting SimulateFromSeed with randomness created with seed 1536821388207640227
Starting the simulation from time Tue Jul 10 10:36:43 UTC 30723, unixtime 907373443003
Simulating... block 9/10, operation 50/70.  --- FAIL: TestStakeWithRandomMessages (0.67s)
	random_simulate_blocks.go:367: tried to delete a nonexistent validator
FAIL

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[rigelrozanski]

oh that one might have been introduced by staking transient store… pretty confused as to why though - I can’t reproduce locally with make test_sim_modules (I’ve tried a couple dozen times) although I did notice it failed non-deterministically on the transient store PR once (#2310)

[ValarDragon]

Its failing on most PR's: https://circleci.com/gh/cosmos/cosmos-sdk/30294?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link, I guess we should make a command so its easy to reproduce the seed for these failing simulations.

[alexanderbez]

@ValarDragon do most of those PRs have develop with transient store support merged in/rebased?

[ValarDragon]

I think so. It seems likely to me that transient store break this, as the failures began after that was merged, and are happening really early on. (On the weekend I was running a couple simulations that got to 1500 blocks, and many that failed at like blocks 100-200, never sub 10 blocks.)

[rigelrozanski]

I'll think we should hold off on heavily investigating this error specifically until we have done the refactors as the update store will be removed at that point (however the transient store should still be getting used).

Unless of course this is due to some new form of non-determinism IN the transient store (which is seems as though it may be? - so maybe actually still valuable to still investigate

[ValarDragon]

There isn't non-determinism. The reason for it only sometimes failing is because gaia_test_sim_modules is set to use a different seed each run. Maybe transient store isn't getting set again properly?

[alexanderbez]

So how should we proceed with this issue?

[rigelrozanski]

still think this is low priority short term, as the solution may be thrown away - But if you're looking for something to do then it may reveal some valuable deeper level bug to do with the transient store. However - there is also a store refactor that joon is working on (which may affect this?) - so yeah probs not the best way to spend time

[cwgoes]

I expect whatever this was has been fixed by #2394 - at minimum, the old code path is no longer relevant.

Reopen if it can be replicated.