STF Design Enhancement Proposal

[clangenb]

I find it cumbersome to make changes to Stf and StfState because it is all entangled with enclave code.

This is just a rough idea, but I believe we can improve this greatly:

The StfState is mostly intertwined with the enclave concerning serialization, crypto and hashing. In my option we could get rid of this by introducing traits in the Stf crate ; SgxDeSerialize, StateCrypto, StateHash (naming to be improved). These traits are then implemented in the enclave. Then we could make the Stf generic over these traits and even handle stuff concerning these three traits under the hood of Stf, which saves a lot of boilerplate code in the enclave. It gets even more elegant if we make Stf a trait.

A snipped of the proposed Stf code follows to give an idea:

pub trait Stf<StfState>
where
    StfState: SgxDeSerialize + StateCrypto + StateHash

{
   fn init_state() -> StfState
   fn update_storage(shard: ShardIdentifier, update_map: HashMap<Vec<u8>> -> StfResult<()>;
   // maybe we can even make that generic over the TrustedCallSigned if it is abstracted as a trait
   fn execute(&mut self, call: TrustedCallSigned) -> StfResult<()>
   ...
}

impl Stf for StfState {
   fn update_storage(shard ....) -> StfResult<()> {
     // deserialize would essentially contain the methods we have in enclave/lib/state currently.
      let state = <Self as SgxDeSerialize>::unseal(shard)?;
     state.execute_with(|| ...)?;
   }

   /// the rest follows a similar procedure...
}

[brenzi]

We should also consider that our two first client use cases don't make use of the stf. It should be very modular, so we can easily leave away he stf (or replace it with things we used for PolkaDEX or Ternoa) @haerdib can you comment if an abstraction of the stf would be helpful here? In the end, PolkaDEX GW is stateful as well, so it could be an stf. But it was too different from our sgx-runtime setup.

[haerdib]

I think that could be really helpful. First of all, with a stf trait we could do better and simpler unit testing. However, for this to cover all possible use cases of state persistence it needs to be extremely abstract. For PolkaDEX we have implemented several different "caches" to represent the states, but none of it actually uses file sealing within the enclave. If necessary, the actual file storage is done outside of the enclave.
For example:

• Market Cache (No file storage at all)
• Create Order Cache (Not file storage at all)
• Balance Storage (File storage will be outside of enclave)
• Order Book Storage (File storage outside of enclave)

The trusted getter get_balance interacts with the balance storage, the trusted call create_order interacts with the create order cache and so on. Maybe PolkaDex is too extreme of an example.. ?
I think some more thoughts are necessary on this topic, starting with the basic question what we actually want to represent with a "state", respective when does a call count as a "state transfer function"?

[brenzi]

what is the status of this now? CC @mullefel

[clangenb]

As far as I know, we have not touched this.

[murerfel]

We have not done any refactoring work on the STF crate yet, this is still open

[clangenb]

Added several '#269' for relevant code that should be considered in this issue.