Parallel transaction processing design

[garious]

• Should we do L0 verification of blocks without context?
  • We can calculate deltas
  • Some event data doesn’t coalesce well (i.e. Witness events)
    • But two timestamps do.
• Should events within a single entry be processed in parallel?
  • How to identify when order affects output?
    • Example: Race(PayAfter(date, to), CancelOnSig(from))
    • Applying a credit first could allow a debit to proceed
  • For conflict resolution, should we look to event order within the entry?
  • When leader creates the entry, should it order ambiguous events or discard one? Discard all?
• Should events across entries (all events in one block) be processed in parallel?
  • For conflict-resolution, entry order within the block unambiguously determines order.

[garious]

Notes from initial discussion with @aeyakovenko:

• First event seen gets added to the entry. Any event that can't be processed in parallel gets pushed back and considered in the following entry.
• Calculating only deltas without any context sounds appealing. Applying the delta can be a separate validation step. Note this is only applicable to validators, where you have the option to reject an entire block. The leader, on the other hand, needs to reject bad transactions individually.
• No intent to parallelize event processing across entries. Note that may massively reduce the opportunity for parallelism. Need to run the numbers on that.

Takeaway: maximize transaction throughput.

[garious]

What I ended up implementing is to put separate read/write locks on each of the following:

• account balance
• the table of all balances
• each pending smart contract
• the table of pending smart contracts
• the set of previous transaction signatures

I also changed transaction processing to apply all debits first, and once complete, start applying all credits. This ensures that the same set can be validated in any order and that no debit will pull the account below zero. The strategy therefore rejects transactions that would otherwise be valid if a credit was applied first.