doc: [en] Add ostracon-specific VRF+BLS feature documents

docs/features/README.md

@@ -0,0 +1,66 @@
+# Ostracon: A Fast, Secure Consensus Layer for The Blockchain of New Token Economy
+
+Version 1.0 :: [日本語](README_ja.md)
+
+## Ostracon Overview
+
+Ostracon is a core-component that provides a Byzantine fault-tolerant (BFT) consensus mechanism for the LINE Blockchain ecosystem. This determines the order of transactions that are executed by applications, then generates and verifies blocks which are containers of transactions.
+
+LINE Blockchain sets out a number of principles to be archived in selecting the technology in order to make the consensus mechanism applicable not only to services on the internet, but also to finance and industry.
+
+**Security**: Completeness and soundness sufficient for practical use, based on cryptographic theory.
+**Consistency**: A consensus algorithm with strong integrity (finality).
+**Fault-Tolerance**: Safety and liveness against system failures, including Byzantine failures.
+**Performance and Scalability**: One block every two seconds with a capability of 1000 TPS or above.
+**Inter-chain Connectivity**: interoperability with other blockchains besides LINE Blockchain.
+
+P2P consensus algorithms based on BFT are more suitable than Bitcoin-like proof of work (PoW) in terms of functionality and performance. Among them, Tendermint-BFT, with its modern blockchain-optimized design, was the closest implementation in our direction (and even better, it can be connected to Cosmos Hub).
+
+We are introducing two new cryptographic technologies with Tendermint-BFT to further improve our blockchain. One is Verifiable Random Function (VRF), which was introduced to randomly select the Proposer node that will generate blocks and makes future selection unpredictable. This randomness is expected to deter malicious attacks and make it difficult for participants to act in collusion at some point in the future.
+
+Another feature is the Boneh–Lynn–Shacham (BLS) signature. BLS signature scheme, which is based on bilinear mapping, gives us the ability to aggregate multiple digital signatures into a single one. In many blockchain protocols, large amounts of signatures must be stored to approve a block. Enabling BLS signature aggregation reduces the footprint and can significantly improve communication overhead and storage consumption.
+
+## Layered Structure
+
+Ostracon includes the Consensus and Networking layers of the three layers that construct a LINE BLockchain node: Application, Consensus, and Networking.
+
+![Layered Structure](layered_structure.png)
+
+Transactions that have not yet been incorporated into a block are shared among nodes by an anti-entropy mechanism (gossipping) in the Network layer called mempool. Here, the Network and Consensus layers consider transactions as simple binaries and don't care about the contents of the data.
+
+## Specifications and Technology Stack
+
+| Specifications        | Policy / Algorithms           | Methods / Implementations                                    |
+| :-------------------- | :---------------------------- | :----------------------------------------------------------- |
+| Participation         | Permissioned                  | Consortium or Private                                        |
+| Election              | Proof of Stake                | VRF-based Weighted Sampling without Replacement + SplitMix64 |
+| Agreement             | Strong Consistency w/Finality | Tendermint-BFT                                               |
+| Signature             | Elliptic Curve Cryptography   | Ed25519, *BLS12-381*<sup>*1</sup>                            |
+| Hash                  | SHA2                          | SHA-256, SHA-512                                             |
+| HSM                   | *N/A*                         | *No support for VRF or signature aggregation*                |
+| Key Auth Protocol     | Station-to-Station            |                                                              |
+| Tx Sharing Protocol   | Gossiping                     | mempool                                                      |
+| Application Protocol  | ABCI                          |                                                              |
+| Interchain Protocol   | IBC (Cosmos Hub)              |                                                              |
+| Storage               | Embedded KVS                  | LevelDB                                                      |
+| Message Recovery      | WAL                           |                                                              |
+| Block Generation Time | 2 seconds                     |                                                              |
+
+<sup>*1</sup> experimental implementation.
+
+## Ostracon Features
+
+* [Extending Tendermint-BFT with VRF-based Election](consensus/README.md)
+* [BLS Signature Aggregation](signature-aggregation/README.md)
+
+## Consideration with Other Consensus Schemes
+
+What consensus schemes are used by other blockchain implementations? We went through a lot of comparison and consideration to determine the direction of Ostracon.
+
+The **PoW** used by Bitcoin and Ethereum is the most well-known consensus mechanism for blockchain. It has a proven track record of working as a public chain but has a structural problem of not being able to guarantee consistency until a sufficient amount of time has passed. This would cause significant problems with lost updates in the short term, and the inability to scale performance in the long term. So we eliminated PoW in the early stages of our consideration.
+
+The consensus algorithm of Tendermint, **Tendermint-BFT**, is a well-considered design for blockchains. The ability to guarantee finality in a short period of time was also a good fit for our direction. On the other hand, the weighted round-robin algorithm used as the election algorithm works deterministically, so participants can know the future Proposer, which makes it easy to find the target and prepare an attack. For this reason, Ostracon uses VRF to make the election unpredictable in order to reduce the likelihood of an attack.
+
+**Algorand** also uses VRF, but in a very different way than we do: at the start of an election, each node generates a VRF random number individually and identifies whether it's a winner of the next Validator or not (it's similar to all nodes tossing a coin at the same time). This is a better way to guarantee cryptographic security while saving a large amount of computation time and power consumption compared to the PoW method of identifying the winner by hash calculation. On the other hand, it's difficult to apply this scheme to our blockchain for several reasons: the number of Validators to be selected is non-deterministic and includes random behavior following a binomial distribution, the protocol complexity increases due to mutual recognition among the winning nodes, and it's impossible to find nodes that have been elected but have sabotaged their roles.
+
+We have considered a number of other consensus mechanisms, but we believe that the current choice is the closest realistic choice for role election and agreement algorithms for P2P distributed systems. However, since Ostracon doesn't have a goal of experimental proofs or demonstrations for any particular research theory, we are ready to adopt better algorithms if they are proposed in the future.

docs/drafts/features/index_ja.md → docs/features/README_ja.md

@@ -1,6 +1,6 @@
 # Ostracon: A Fast, Secure Consensus Layer for The Blockchain of New Token Economy
 
-Version 1.0 :: [English](index.md)
+Version 1.0 :: [English](README.md)
 
 ## Ostracon Overview
 
@@ -62,8 +62,8 @@ Networking レイヤーが含まれています。
 
 ## Ostracon Features
 
-* [Extending Tendermint-BFT with VRF-based Election](consensus_ja.md)
-* [BLS Signature Aggregation](signature_aggregation_ja.md)
+* [Extending Tendermint-BFT with VRF-based Election](consensus/README_ja.md)
+* [BLS Signature Aggregation](signature-aggregation/README_ja.md)
 
 ## Consideration with Other Consensus Schemes
 

docs/features/consensus/README.md

@@ -0,0 +1,86 @@
+# Extending Tendermint-BFT with VRF-based Election
+
+## Consensus Overview
+
+Ostracon's block generation mechanism based on Tendermint-BFT consists of the following three phases. We here refer to the generations of blocks as *height*, and a single approval round consisting of the following three processes as *round*.
+
+**Election**. Elect one Proposer and several Voters from a candidate node set. This is the same as a Leader Election in a general distributed system, but in blockchain, it must be designed to prevent artificial selection so that malicious interference doesn't degrade the overall performance of the system. Also note that there is no centralized authority involved in Ostracon elections to ensure fairness. Since the election results can be computed deterministically by all nodes, each node can autonomously determine whether it has been elected as a Proposer or Voter.
+
+**Block Generation**. The elected Proposer proposes a block. Unapproved transactions that have not yet been included in the blockchain are shared among nodes in the network via P2P and stored in an area of each node called the mempool. The node selected as the Proposer generates a block from the unapproved transactions remaining in its mempool and proposes it to the Voters.
+
+**Block Verification**. The block proposed by the Proposer is verified by elected Voters. Each Voter votes on whether the block is correct or not, and the votes are replicated by Tendermint-BFT to the other Voters, and if more than 2/3+1 of all Voters vote in favor of the block, the block is officially approved. On the other hand, if a quorum is not reached, the proposed block is rejected and a new round of elections or voting is started over (Tendermint-BFT has several shortcuts depending on the reason for rejection).
+
+![VRF-based Block Generation Round](vrf_based_round.png)
+
+## VRF-based Consensus Group Election
+
+VRF is an algorithm for generating a hash value $t$ that can be used as a cryptographic pseudo-random number. The differences between VRF and typical hash functions or pseudo-random number generators are that only the owner of the private key can generate the hash value $t$, and anyone with the corresponding public key can verify the correctness of the hash value.
+
+A VRF hash generator $k$ generates a proof $\pi$ (VRF Proof) from the message $m$ using its private key $S_k$ as in Equation (1). Here, the hash value $t$ can be acquired from the proof $pi$ using Equation. (2). On the other hand, to verify that the hash value $t$ was generated by the owner of the private key $S_k$ based on the message $m$, the verifier applies the public key $P_k$ for $S_k$, $m$, and $\pi$ to Equation (3) to verify that both hash values are identical.
+
+![VRF Expression](math_expression.png)
+
+```math
+\begin{eqnarray}
+\pi & = & {\rm vrf\_prove}(S_k, m) \\
+t & = & {\rm vrf\_proof\_to\_hash}(\pi)
+\end{eqnarray}
+\begin{equation}
+{\rm vrf\_proof\_to\_hash}(\pi) \overset{\text{?}}{=} {\rm vrf\_verify}(P_k, m, \pi)
+\end{equation}
+```
+
+With Ostracon, the Proposer and Voters of the next block are selected randomly by a verifiable random number from the Proposer that created the previous block. A VRF Proof field $pi$ is added to the block for this purpose.
+
+The node that receives the new block initiates the election phase. In this phase, it verifies the VRF Proof $\pi$ contained in the block, calculates the VRF hash $t$, which is a "fair pseudo-random number," and selects the Proposer and Voters for this round based on that value. This is done by a simple and fast weighted random sampling based on the probability of selection according to Stake holdings (in other words, based on PoS).
+
+![VRF-based Proposer/Voter Election](vrf_election.png)
+
+The node selected as the Proposer by this phase picks up the unapproved transactions from its own mempool and generates a proposal block (at this point, the block is not confirmed yet). Then, the Proposer calculates VRF Proof $\pi'$ using the previous VRF Hash $t$ that selected itself, the new block height $h$, and the current round $r$ and sets it to the block.
+
+![VRF Prove](math_prove.png)
+
+```math
+\begin{eqnarray*}
+m_h & = & {\rm SHA256}(h \,\|\, r \,\|\, t_{h-1}) \\
+\pi_h & = & {\rm vrf\_prove}(S_i, m_h) \\
+t_h & = & {\rm vrf\_proof\_to\_hash}(\pi_h)
+\end{eqnarray*}
+```
+
+Note that the message $m$ used to calculate the new VRF Proof $\pi$ doesn't involve the hash value of the block itself. We consider that the hash value of the block is inherently insecure because the Proposer who generates the block can obtain a favorable value by trial and error.
+
+![VRF-based Block Generation](vrf_block_generation.png)
+
+A node that is selected as a Voter in the election phase verifies the received Proposal block and votes on it. The votes are replicated by Tendermint-BFT through prevote, precommit, and commit. The block is confirmed if more than a quorum of valid votes are collected.
+
+![VRF-based Block Validation](vrf_block_validation.png)
+
+During the verification phase, the following VRF-related verifications are performed in addition to block verification:
+
+* The Proposer that generated the block must be a node selected based on the VRF hash of its previous block. This can be determined by matching the node that actually generated the block with the Proposer selected by weighted random sampling using the VRF hash $t$.
+* The $\pi$ contained in the block must be a VRF Proof generated using the private key of the Proposer. If the $t$ calculated from the VRF Proof $\pi$ matches the $t$ calculated using the `vrf_verify()` function, we can conclude that $\pi$ is not forged.
+
+![VRF Verify](math_verify.png)
+
+```math
+{\rm vrf\_verify}(P_i, m_h, \pi_h) \overset{\text{?}}{=} {\rm vrf\_proof\_to\_hash}(\pi_h)
+```
+
+By repeating this sequence of rounds, fair random sampling can be chained across all block generation.
+
+![BFT-based Block Generation](bft_round.png)
+
+Recall here that the node that receives the block can deterministically calculate which nodes are the next Proposer and Voters. By revealing the nodes that are responsible for generating and verifying blocks in a given round, we can penalize nodes that are elected but don't actually perform their responsibility or that behave malicious actions such as Eclipse attacks. On the other hand, it's still difficult to predict the Proposer and Voters beyond one block, as they are only revealed for the minimum necessary time.
+
+VRF is currently implemented using Ed25519, and even if a node uses BLS signatures, it also has an Ed25519 key to calculate VRF.
+
+## Voters
+
+In the Ostracon network, Validators mean candidate nodes that hold Stakes and can be elected as Proposers or Voters. The Voters (a subset of Validators) are a new concept introduced in Ostracon for two reasons; first, to make the distribution of rewards to nodes elected as Voters flexible, and second, to allow the ratio of Byzantine assumptions to be changed in networks with different trust policies for the participant nodes (as a result of the configuration, if the number of Voters is set to match the number of Validators, the behavior will be the same as in Tendermint).
+
+Voter selections use a pseudo-random function $r$ to generate a sequence of random numbers in order to randomly select multiple nodes from a single VRF hash $t$. It's more important that $r$ is fast, simple to implement, has no variant by different interpretations, and saves memory since $t$ already has the properties of a cryptographic pseudo-random number. Ostracon uses a fast shift-register type pseudo-random number generation algorithm, called SplitMix64, for Voter selection.
+
+## Disciplinary Scheme for Failures
+
+Although Ostracon's consensus scheme works correctly even if a few nodes fail, it's ideal that failed nodes aren't selected for the consensus group in order to avoid wasting network and CPU resources. In particular, for cases that aren't caused by general asynchronous messaging problems, such as intentional malpractice, evidence of the behavior (whether malicious or not) will be shared and action will be taken to eliminate the candidate from the selection process by forfeiting the Stake.

docs/drafts/features/consensus_ja.md → docs/features/consensus/README_ja.md

@@ -104,9 +104,9 @@ VRF は現在のところ Ed25519 鍵を使用して実装されています。B
 ## Voters
 
 Ostracon のネットワークでは Stake を保有して Proposer または Voter に選出される可能性のある候補ノードを Validator としています。
-Voter は 2 つの理由で Ostracon に新しく導入された概念です; 一つ目は Voter に選出されたノードへの報酬配当を調整するため、もう一つは
-参加可能なノードの信頼ポリシーが異なるネットワークでビザンチンとして想定する比率を調整可能にするためです (Voter 数が Validator 数と
-一致する設定では Tendermint と完全に一致します)。
+Voter は Validator のサブセットであり、2 つの理由で Ostracon に新しく導入された概念です; 一つ目は Voter に選出されたノードへの
+報酬配当を調整するため、もう一つは参加可能なノードの信頼ポリシーが異なるネットワークでビザンチンとして想定する比率を調整可能にするためです
+(Voter 数が Validator 数と一致する設定では Tendermint と完全に一致します)。
 
 Voter 選出では、一つの VRF ハッシュ $t$ から複数のノードをランダムに選択するために、疑似乱数関数 $r$ を使って $t$ に基づいた乱数列を
 生成します。$t$ が既に暗号論的疑似乱数の性質を持つことから、この $r$ は実装がシンプルで変種が発生しづらく、高速で省メモリであることが