Automatic checking try_state hooks in tests

[ggwpez]

Currently we have this setup of a try_state hook per pallet which can check storage invariants.
This is very good, but the hook has to be invoked manually in tests or placed at the end once in a modified new_test_ext.

It would be nice if we could check these invariants more often. For example on every 100th storage modification.
One way to achieve this could be to hook into the storage functions of TestExternalities and register a post-modify hook which can then be set to try_state. Do you think that would work?

[bkchr]

We could also build a special runtime with try_state enabled and let the node run with this runtime as override. Then we have some external tool that triggers the try_state runtime call on every nth block import.

[xlc]

The external tool could be chopsticks. AcalaNetwork/chopsticks#70

[ggwpez]

We could also build a special runtime with try_state enabled and let the node run with this runtime as override. Then we have some external tool that triggers the try_state runtime call on every nth block import.

This is exactly what try-runtime follow-chain does if I am not mistaken. It loops through the pallets and try_states them on each new imported block.

[bkchr]

Then this issue would be solved?

[ggwpez]

Then this issue would be solved?

This issue is for checking the try_state inside of unit tests. The follow-chain is for actually running a live node.

[bkchr]

Ahh now I get your comment "on every 100th storage modification". Hmm. This could be breaking stuff as an invariant is only required to hold after a function has finished or even multiple. I think the idea to check try_state after each test is much better.

[kianenigma]

We could also build a special runtime with try_state enabled and let the node run with this runtime as override. Then we have some external tool that triggers the try_state runtime call on every nth block import.

paritytech/substrate#13563 waiting to see the SDK team close this sir @bkchr :D

[kianenigma]

For tests, we don't have a good way to do this. Perhaps someday we can create a framework for building test setup that does this for us. Think constuct_runtime_with_ext_builder. This will generate a shit ton of other code for you, including a fn build_and_execute that will call try_state automatically.

Also, as I am grooming issues today, if we have #261, then we can automatically call try_state on post_dispatch given #[cfg(test)]

Lastly, perhaps we can leverage impl Drop of ExtBuilder or TestExternalities to achieve this.