Skip to content
This repository has been archived by the owner on Jun 11, 2024. It is now read-only.

Latest commit

 

History

History
189 lines (129 loc) · 11.1 KB

lip-0002.md

File metadata and controls

189 lines (129 loc) · 11.1 KB
LIP: 0002
Title: Change to byte based block size limit
Author: Iker Alustiza <iker@lightcurve.io>
        Nazar Hussain <nazar@lightcurve.io>
Discussions-To: https://research.lisk.com/t/change-to-byte-based-block-size-limit/
Status: Replaced
Type: Standards Track
Created: 2018-08-17
Updated: 2024-01-04
Superseded-By: 0055

Abstract

This LIP proposes to replace the literal 25 transactions per block size limit with a byte based block size limit.

Copyright

This LIP is licensed under the Creative Commons Zero 1.0 Universal.

Motivation

As the Lisk protocol evolves and gains popularity, a clear increase in usage and activity in the network is expected. To cater for this increased demand a more flexible and higher throughput is required. The current maximum of 25 transactions per block imposes a limit that is based on a literal number of transactions per block. This limitation method also conflicts with the varying byte size of each transaction type that can be processed within any given block. We therefore propose a byte based block size limit.

Rationale

We propose to set a maximum block size of 15 KiB (15 * 1024 bytes). This size will maintain the yearly blockchain growth below 50 GiB, even when blocks are always at full capacity, and the active delegates do not miss any blocks. The summary of the numerical implications are as follows:

  • Maximum block size: 15 KiB.
  • The block history will grow around 45 GiB per year (approximately 66024365.2515/(1024*1024) GiB) assuming full blocks.
  • According to the transaction sizes given in the protocol documentation, each transaction type will use the following space in a block:
Tx type Block usage (%)
0 - Transfer (basic) 0.78%
0 - Transfer (2nd signature) 1.2%
0 - Transfer (2nd + data field) 1.63%
1 - Second Signature 1%
2 - Delegate 0.91%
3 - Vote (33 votes) 15.08%
4 - Multisignature (16 signatures) 8.15%
5 - Dapp 1.66%
6 - InTransfer 1.33%
7 - OutTransfer 1.33%

  • Given the previous numbers stated, each block will be able to accommodate:

    • 6 votes and [ 8 tx(a) or 5 tx(b) or 4 tx(c) ]
    • 4 votes and [ 48 tx(a) or 31 tx(b) or 23 tx(c) ]
    • 2 votes and [ 88 tx(a) or 57 tx(b) or 42 tx(c) ]
    • 128 tx(a) or 82 tx(b) or 61 tx(c)

Where tx(a) is a type 0 - Transfer (basic) transaction, tx(b) is a type 0 - Transfer (2nd signature) transaction, and tx(c) is a type 0 - Transfer (2nd + data field) transaction. A type 0 - Transfer (2nd signature) transaction and a 0 - Transfer (basic + data field) transaction are almost the same size, therefore only one is considered. Here votes are assumed to be a transaction containing 33 votes.

  • 1000 users will require at least 27 minutes (166 blocks) to change a third of their votes (33 votes per vote transaction). They will require approximately 90 minutes to change all of their votes.

To get this data, the size of the block header (112 bytes) is ignored, which can imply an extra 300 MiB of yearly blockchain growth. We have excluded other transaction types from the previous table because they are either one-time transactions per account (delegate registration, second signature registration, etc), or seldom used transaction types (dapp registration, in/out transfer). In the latter case, depending on future development of these transaction types, they can become more common, but the size is assumed to be equivalent to a type 0 - Transfer (basic) transaction.

Moreover, we can compare the maximum transaction throughput per day for current and proposed implementations:

Throughput Comparison (Assuming Full Blocks)

Tx type Tx per day (current) Tx per day (after) Change
0 - Transfer (basic) 216,000 1,080,000 +500%
0 - Transfer (2nd + data field) 216,000 589,090 +272%
1 - Second signature 216,000 869,798 +402%
2 - Delegate 216,000 644,776 +298%
3 - Vote 216,000 55,717 -387%
4 - Multisignature 216,000 105,968 +49%

Transaction Ratio Per Block Comparison

To support the previous table, it is also interesting to take note of historical data from the Lisk mainnet:

Tx Type Tx Count (Current Avg) Ratio Proposed Capacity
0 - Transfer 2.04184768 95.422% 65
1 - Second signature 0.00438445 0.205% 0.21
2 - Delegate 0.00200557 0.094% 0.07
3 - Vote 0.09154353 4.278% 0.28
4 - Multisignature 0.00001629 0.001% 0.00012264
5 - Dapp 0.00000376 0% 0.0
6 - InTransfer 0.00000000 0% 0.0
7 - OutTransfer 0.00000251 0% 0.0

Where we can see that type 0 - Transfer transactions account for more than 95% of all transactions in the blockchain. This implies that, on average and assuming full blocks, 65 balance transactions with 2nd signature and full data field will be included in each block (proposed capacity) in the future.

We believe that given these numbers, 15 KiB is the most adequate choice in terms of network requirements (node bandwidth, blockchain growth, etc) vs network capabilities (transaction per second, votes per hour, etc). Moreover, it could be easily extended in the future if the network usage requires it.

Specification

General Implementation Logic

  • The payload of a block must not weigh more than 15 KiB (15 * 1024 bytes = 15360 bytes). If a delegate generates a bigger block, it should be deemed invalid.

  • When filling a new block with unconfirmed transactions from the pool, the node should optimize the usage of the available space in the block versus available transactions in the pool. For example, when there is less than 2 KiB available for the next block (a vote transaction is 2.3 KiB), the delegate should fill the remaining space using smaller transactions, even though the next transaction in the queue in terms of fee or relative age would be a vote transaction. In this LIP, we propose a basic algorithm to cater for this scenario (please refer to 3. of Implementation Details. However, it is for each delegate to decide whether to implement a different algorithm should they believe it to be more optimal.

  • When a delegate generates a block, it should fetch the unconfirmed transactions from the pool and verify their size again.

Implementation Details

  1. Add getListByteSize to logic.transactionPool

    • Maintain a static counter for size and update it on every call of the following methods add<Queue>Transaction, remove<Queue>Transaction. For example, addUnconfirmedTransaction and removeUnconfirmedTransaction functions.

    • Or call in realtime the getBytes function on each transaction within the queue.

  2. Add getListItem(index = 0) to logic.transactionPool which will return a transaction within the list at a given index. By default, it should return the transaction at the top of the list.

  3. Reimplement the fillPool function in logic.transactionPool using the proposed algorithm.

  4. Update modules.blocks.process.generateBlock:

    • Call modules.transactions.getUnconfirmedTransactionList without a limit to get all the unconfirmed transactions. As those transactions should always be approximately equal to 15 KiB.

  5. Update constants.maxPayloadLength to 15 KiB.

  6. Remove constants.maxTxsPerBlock from constants.js.

  7. Remove the redundant numberOfTransactions check from blocks verification for historical data, as it is already included in block's signature.

  8. Remove the redundant payloadLength check from blocks verification for historical data, as it is already included in block's signature.

  9. Add maxPayloadLength checks up to 15 KiB for recent blocks.

Algorithm for logic.transactionPool#fillPool

algorithm fillPool
	input: No
	output: No

	Set unconfirmedSize to logic.transactionPool.getListByteSize('Unconfirmed')
	Set newTransactions to empty array

	While unconfirmedSize < 15 KiB do
		Call pickTransactionFromList('Multisignature', 15 KiB - unconfirmedSize)
		Push transaction to newTransactions Array
		Set unconfirmedSize = unconfirmedSize + transaction.getBytes()

	While unconfirmedSize < 15 KiB do
		Call pickTransactionFromList('Queued', 15 KiB - unconfirmedSize)
		Push transaction to newTransactions Array
		Set unconfirmedSize = unconfirmedSize + transaction.getBytes()

	Call applyUnconfirmedList(newTransactions)

	return

procedure pickTransactionFromList
	input: listName - Name of the list from which to pick transaction
	input: maxAllowedByteSize - Maximum byte size of transaction to pick
	input: skipIndex - Skip the top N transactions default to 0
	output: selectedTransaction

	Set count to logic.transactionPool.count<listName>()

	If count <= skipIndex
		return null

	Set selectedTransaction to getListItem(listName, skipIndex)

	If selectedTransaction.getBytes() < maxAllowedByteSize
	  Call logic.transactionPool.remove<listName>Transaction(selectedTransaction.id)
		return selectedTransaction
	Else
		Call pickTransactionForPool(listName, maxAllowedByteSize, skipIndex + 1)

Related Implementation

For completeness, we add here a related implementation to this proposal which consists of refactoring the transactionPool for multisignature transactions:

  1. Only pending multisignature transactions should stay in the multisignature list.
  2. As soon as the multisignature transaction is ready it should be pushed to queued list.
  3. In the previous step, we only look into the queued list to fill the pool while generating blocks.

Backwards Compatibility

  • This proposal will cause a hard fork in the network.
  • Once implemented, blocks with more than 25 transactions will be valid, and blocks with more than 7 voting transactions will be invalid.
  • The proposed changes will not imply any change in the blocks schema. It will contain the same fields as the current implementation.
  • There is no need to add any exception validation logic, as the numberOfTransaction property is already part of the block signature. So validating previous blocks will work before.

Reference Implementation

TBD