Rewrite transaction pool

[Isabello]

Belongs to: #449

Improve transaction pool efficiency. No longer apply transactions in an unconfirmed state.

Task List:

Some parts are already covered in #544 and #597, however we still need to rewrite transaction pool - it should be central point of dealing with transactions - so those transactions should go to pool and pool can (or not) initiate validations.

• Transaction pool can contain only unconfirmed transactions - signed (normal txs) or unsigned/partially signed (multisig txs waiting for sign), main object should contains two lists: unverified and verified. verified object should be split into pending and ready.
  • Transaction pool should provide public add method - used when we receive new transaction:
    • Prior schema should be checked (maybe outside txs pool), if bad - discard
    • id should be checked if exists in unverified or verified.pending or verified.ready, if we already have that tx - discard
    • Add tx to unverified
• Write a function that will constantly pull transactions from unverified, perform verifications and if successful push to either verified.pending (when tx is multisign or timestamp is in future) or verified.ready otherwise. Such a worker would be easily replaced with child process in future. That worker should also verify incoming signatures (multisig txs signs), and take care about moving transactions from verified.pending to verified.ready. Worker can run in loop or can be added to jobsQueue (interval to research).
  • As stated in Redesign accounts ledger #544 - optionally provide a worker to check transactions against confirmed states - to research.
  • Because transactions will be checked against confirmed state when forging - transaction pool should provide public delete method that will delete failed transaction from the pool.
  • Worker should expire transactions
• On new block (after is verified) transactions included in that block should be deleted from pool via delete method (we not immediatelly delete transactions that we included in block when forge, because that block can fail).
• When we need to rollback and delete existing block - we should first retrieve txs included in that block and after successful block deletion - push those txs back to transaction pool verified.ready via rollback/addReady method.
• Transaction pool should also provide method to pack transactions to binary format for fast broadcast - to research, reference Improve block propagation #541.
• Write method getReady that will select transactions that are ready to be included in block (it should require count/limit parameter) from verified.ready.
  • We need transactions to be sorted depending on fee and then received time (not transaction timestamp!), there are some options - we should pick one with best performance:
    • Worker can keep verified.ready list sorted (resort after new/delete txs) - only if node is forging.
    • List can be iterated when required and best transactions can be selected (req min 25 iterations per block) - that one should be fastest.
• Implement limit for number of transactions that pool can store - adjustable in config.
• Implement other public methods:
  • getUsage that will return counters for unverified, verified.pending and verified.ready
  • get that will return transaction by id
  • getAll that will return all transactions, with optional limited filters (such as unverified, pending, ready, sender_id, sender_pk, recipient_id, recipient_pk) - to research if we need more here.
  • checkBalance (optional) - will return estimated balance for requested account - it should get account from database and apply to balance all transactions that are stored in pool and affect that account

[4miners]

Added task list to issue description.

[4miners]

#457 should be done first.