doc: [en] Add ostracon-specific VRF+BLS feature documents (Finschia#294)

docs/drafts/features/consensus.md

@@ -0,0 +1,143 @@
+# Extending Tendermint-BFT with VRF-based Election
+
+## Consensus Overview
+
+Ostracon's block generation mechanism based on Tendermint-BFT consists of the following three phases. We here refer to
+the block generation as *height*, and a single approval round consisting of three processes as *round*.
+
+Ostracon's block generation mechanism based on Tendermint-BFT consists of the following three phases. We here refer to
+the number of block generation as *height*, and a single approval round consisting of the following three processes
+as *round*.
+
+**Election**. Elect one Proposer and several Voters from candidate node set. This is the same as a Leader Election
+in a general distributed system, but in blockchain, it must be designed to prevent artificial selection so that
+malicious interference doesn't degrade the overall performance of the system. Also note that there is no centralized
+authority involved in Ostracon elections to ensure fairness. Since the election results can be computed
+deterministically by all nodes, each node can autonomously determine whether it has been elected as a Proposer or Voter.
+
+**Block Generation**. The elected Proposer proposes a block. Unapproved transactions that have not yet been
+included in the blockchain are shared among nodes in the network via P2P and stored in an area called mempool
+of each node. The node selected as the Proposer generates a block from the unapproved transactions remaining in
+its mempool and proposes it to the Voters.
+
+**Block Verification**. The block proposed by the Proposer is verified by elected Voters. Each Voter votes on whether
+the block is correct or not, and the votes are replicated by Tendermint-BFT to the other Voters, and if more than
+2/3+1 of all Voters vote in favor of the block, the block is officially approved. On the other hand, if a quorum
+is not reached, the proposed block is rejected and a new round of elections or voting is started over (Tendermint-BFT
+has several shortcuts depending on the reason for rejection).
+
+![VRF-based Block Generation Round](vrf_based_round.png)
+
+## VRF-based Consensus Group Election
+
+VRF is an algorithm for generating a hash value $t$ that can be used as a cryptographic pseudo-random number.
+The differences between VRF and typical hash functions or pseudo-random number generators are that only the owner
+of the private key can generate the hash value $t$, and anyone with the corresponding public key can verify
+the correctness of the hash value.
+
+A VRF hash generator $k$ generates a proof $\pi$ (VRF Proof) from the message $m$ using its private key $S_k$
+as in Equation (1). Here, the hash value $t$ can be acquired from the proof $pi$ using Equation. (2). On the other hand,
+to verify that the hash value $t$ was generated by the owner of the private key $S_k$ based on the message $m$,
+the verifier applies the public key $P_k$ for $S_k$, $m$, and $\pi$ to Equation (3) to verify that both hash values
+are identical.
+
+
+![VRF Expression](math_expression.png)
+
+```math
+\begin{eqnarray}
+\pi & = & {\rm vrf\_prove}(S_k, m) \\
+t & = & {\rm vrf\_proof\_to\_hash}(\pi)
+\end{eqnarray}
+\begin{equation}
+{\rm vrf\_proof\_to\_hash}(\pi) \overset{\text{?}}{=} {\rm vrf\_verify}(P_k, m, \pi)
+\end{equation}
+```
+
+In Ostracon, the Proposer and Voters of the next block are selected randomly by a verifiable random number from
+the Proposer that created the previous block. A VRF Proof field $pi$ is being added to the block for this purpose.
+
+The node that receives the new block initiates the election phase. In 　this phase, it verifies the VRF Proof $\pi$
+contained in the block, calculates the VRF hash $t$, which is a "fair pseudo-random number," and selects the Proposer
+and Voters for this round based on that value. This is done by a simple and fast weighted random sampling based on
+the probability of selection according to Stake holdings (i.e., based on PoS).
+
+![VRF-based Proposer/Voter Election](vrf_election.png)
+
+The node selected as the Proposer by this phase picks up the unapproved transactions from its own mempool and generates
+a proposal block (at this point, the block is not confirmed yet). Then, the Proposer calculates a VRF Proof $\pi'$
+using the previous VRF Hash $t$ that selected itself, the new block height $h$, and the current round $r$ and sets it
+to the block.
+
+
+![VRF Prove](math_prove.png)
+
+```math
+\begin{eqnarray*}
+m_h & = & {\rm SHA256}(h \,\|\, r \,\|\, t_{h-1}) \\
+\pi_h & = & {\rm vrf\_prove}(S_i, m_h) \\
+t_h & = & {\rm vrf\_proof\_to\_hash}(\pi_h)
+\end{eqnarray*}
+```
+
+Note that the message $m$ used to calculate the new VRF Proof $\pi$ doesn't involve the hash value of the block itself.
+We consider that the hash value of the block is inherently insecure because the Proposer who generates the block can
+obtain a favorable value by trial and error.
+
+![VRF-based Block Generation](vrf_block_generation.png)
+
+A node that is selected as a Voter in the election phase verifies the received Proposal block and votes on it.
+The votes are replicated by Tendermint-BFT through prevote, precommit, and commit, and the block is confirmed
+if more than a quorum of valid votes are collected.
+
+![VRF-based Block Validation](vrf_block_validation.png)
+
+During the verification phase, the following VRF-related verifications are performed in addition to block verification:
+
+1. The Proposer that generated the block must be a node selected based on the VRF hash of its previous block.
+   This can be determined by matching the node that actually generated the block with the Proposer selected by
+   weighted random sampling using the VRF hash $t$.
+2. The $\pi$ contained in the block must be a VRF Proof generated using the private key of the Proposer. If the $t$
+   calculated from the VRF Proof $\pi$ matches the $t$ calculated using the `vrf_verify()` function, we can conclude
+   that $\pi$ is not forged.
+
+![VRF Verify](math_verify.png)
+
+```math
+{\rm vrf\_verify}(P_i, m_h, \pi_h) \overset{\text{?}}{=} {\rm vrf\_proof\_to\_hash}(\pi_h)
+```
+
+By repeating this sequence of rounds, fair random sampling can be chained across all block generation.
+
+![BFT-based Block Generation](bft_round.png)
+
+Recall here that the node that receives the block can deterministically calculate which nodes are the next Proposer
+and Voters. By revealing the nodes that are responsible for generating and verifying blocks in a given round, we can
+punish nodes that are elected but don't actually perform their responsibility or that behave malicious actions such as
+Eclipse attacks. On the other hand, it's still difficult to predict the Proposer and Voters beyond one block, as they
+are only revealed for the minimum necessary time.
+
+VRF is currently implemented using Ed25519, and even if a node uses BLS signatures, it also has an Ed25519 key to
+calculate VRF.
+
+## Voters
+
+In the Ostracon network, Validators mean candidate nodes that hold Stakes and can be elected as Proposers or Voters.
+The Voters are a subset of Validators are a new concept introduced in Ostracon for two reasons; first, to make flexible
+the distribution of rewards to nodes elected as Voters, and second, to allow the ratio of Byzantine assumptions to be
+changed in networks with different trust policies for the participant nodes (as a result of the configuration, if the
+number of Voters is set to match the number of Validators, the behavior will be the same as in Tendermint).
+
+Voters selection uses a pseudo-random function $r$ to generate a sequence of random numbers in order to randomly
+select multiple nodes from a single VRF hash $t$. It's more important that $r$ is simple to implement, no variant
+by different interpretations, fast, and memory-saving since $t$ already has the properties of a cryptographic
+pseudo-random number. The Ostracon uses a fast shift-register type pseudo-random number generation algorithm,
+called SplitMix64, for this Voters selection.
+
+## Disciplinary Scheme for Failures
+
+Although Ostracon's consensus scheme works correctly even if a few nodes fail, it's ideal that failed nodes aren't
+selected for the consensus group in order to avoid wasting network and CPU resources. In particular, for cases that
+aren't caused by general asynchronous messaging problems, i.e., intentional malpractice,  evidence of the behavior
+(whether malicious or not) will be shared and action will be taken to eliminate the candidate from the selection
+process by forfeiting the Stake.

docs/drafts/features/consensus_ja.md

@@ -104,9 +104,9 @@ VRF は現在のところ Ed25519 鍵を使用して実装されています。B
 ## Voters
 
 Ostracon のネットワークでは Stake を保有して Proposer または Voter に選出される可能性のある候補ノードを Validator としています。
-Voter は 2 つの理由で Ostracon に新しく導入された概念です; 一つ目は Voter に選出されたノードへの報酬配当を調整するため、もう一つは
-参加可能なノードの信頼ポリシーが異なるネットワークでビザンチンとして想定する比率を調整可能にするためです (Voter 数が Validator 数と
-一致する設定では Tendermint と完全に一致します)。
+Voter は Validator のサブセットであり、2 つの理由で Ostracon に新しく導入された概念です; 一つ目は Voter に選出されたノードへの
+報酬配当を調整するため、もう一つは参加可能なノードの信頼ポリシーが異なるネットワークでビザンチンとして想定する比率を調整可能にするためです
+(Voter 数が Validator 数と一致する設定では Tendermint と完全に一致します)。
 
 Voter 選出では、一つの VRF ハッシュ $t$ から複数のノードをランダムに選択するために、疑似乱数関数 $r$ を使って $t$ に基づいた乱数列を
 生成します。$t$ が既に暗号論的疑似乱数の性質を持つことから、この $r$ は実装がシンプルで変種が発生しづらく、高速で省メモリであることが

docs/drafts/features/index.md

@@ -1,5 +1,98 @@
 # Ostracon: A Fast, Secure Consensus Layer for The Blockchain of New Token Economy
 
-[日本語](index_ja.md)
+Version 1.0 :: [日本語](index_ja.md)
 
-(WIP)
+## Ostracon Overview
+
+Ostracon is a core-component that provides Byzantine fault-tolerant consensus mechanism for our LINE Blockchain
+ecosystem. This determines the order of transactions that are executed by applications, and generates, verifies
+blocks which are containers of transaction.
+
+LINE Blockchain sets out a number of principles to be archived in selecting the technology in order to make the
+consensus mechanism applicable not only to services on the Internet, but also to finance and industry.
+
+**Security**: Completeness and soundness sufficient for practical use, based on cryptographic theory.
+**Consistency**: A consensus algorithm with strong integrity (finality).
+**Fault-Tolerance**: Safety and liveness against system failures, including Byzantine failures.
+**Performance and Scalability**: One block every two seconds with a capability of 1000TPS+.
+**Inter-chain Connectivity**: interoperability with other blockchains besides LINE Blockchain.
+
+P2P consensus algorithms based on BFT (Byzantine Fault-Tolerance) are more suitable than Bitcoin-like Proof of Work
+in terms of functionality and performance. Amongh them, Tendermint-BFT, with its modern blockchain-optimized design,
+was the closest implementation ot our direction (and even better, it can be connected to Cosmos Hub).
+
+We are introducing two new cryptographic technology with Tendermint-BFT to further improve our blockchain. One is
+Verifiable Random Function (VRF), which was introduced to select the Proposer node that generates a block
+randomly and to make future selection unpredictable. This randomness is expected to give malicious adversaries to
+prepare an attack or to make it more difficult for participants to act in collusion at some point in the future.
+
+Another feature is the BLS signature. BLS signature, which is based on bilinear mapping, have the ability to aggregate
+multiple digital signatures into a single one. In many blockchain protocols, ton of signatures must be stored to
+approve a block. Enabling BLS signature aggregation reduces the footprint and can significantly improve communication
+overhead and storage consumption.
+
+## Layered Structure
+
+Ostracon includes the Consensus and Networking layers of the three layers that constructs a LINE BLockchain node:
+Application, Consensus and Networking.
+
+![Layered Structure](layered_structure.png)
+
+Transactions that have not yet been incorporated into a block are shared among nodes by an anti-entropy mechanism
+(gossipping) in the Network layer called mempool. Here, the Network and Consensus layers consider transactions as
+simple binaries and don't care about the content of the data.
+
+## Specifications and Technology Stack
+
+| Specifications        | Policy / Algorithms           | Methods / Implementations                       |
+|:----------------------|:------------------------------|:------------------------------------------------|
+| Participation         | Permissioned                  | Consortium or Private                           |
+| Election              | Proof of Stake                | VRF-based Weighted Sampling without Replacement + SplitMix64 |
+| Agreement             | Strong Consistency w/Finality | Tendermint-BFT                                  |
+| Signature             | Elliptic Curve Cryptography   | Ed25519, *BLS12-381*<sup>*1</sup>               |
+| Hash                  | SHA2                          | SHA-256, SHA-512                                |
+| HSM                   | *N/A*                         | *No support for VRF or signature aggregation*   |
+| Key Auth Protocol     | Station-to-Station            | |
+| Tx Sharing Protocol   | Gossiping                     | mempool |
+| Application Protocol  | ABCI                          | |
+| Interchain Protocol   | IBC (Cosmos Hub)              | |
+| Storage               | Embedded KVS                  | LevelDB |
+| Message Recovery      | WAL | |
+| Block Generation Time | 2 seconds | |
+
+<sup>*1</sup> experimental implementation.
+
+## Ostracon Features
+
+* [Extending Tendermint-BFT with VRF-based Election](consensus.md)
+* [BLS Signature Aggregation](signature_aggregation.md)
+
+## Consideration with Other Consensus Schemes
+
+What consensus schemes are used by other blockchain implementations? We did a lot of comparison and consideration to
+determine the direction of Ostracon.
+
+The **PoW** used by Bitcoin and Ethereum is the most well-known consensus mechanism for blockchain. It has a proven
+track record of working as a public chain but has a structural problem of not being able to guarantee consistency until
+a sufficient among of time has passed. This would cause significant problems with lost updates in the short term,
+and the inability to scale performance in the long term. So we eliminated PoW in the early stages of our consideration.
+
+The consensus algorithm of Tendermint, **Tendermint-BFT**, is a well-considered design for blockchains. The ability
+to guarantee finality in a short period of time was also a good fit for our direction. On the other hand, the weighted
+round-robin algorithm used as the election algorithm works deterministically, so participants can know the future
+Proposer, which makes it easy to find the target and prepare an attack. For this reason, Ostracon uses VRF to make
+the election unpredictable in order to reduce the likelihood of an attack.
+
+**Algorand** also uses VRF, but it's a very different way than we do: at the start of an election, each node generates
+a VRF random number individually and identifies itself as a winner of the next Validator or not (it's similar to all
+nodes tossing a coin at the same time). This is a better way to guarantee cryptographic security while saving a large
+amount of computation time and power consumption compared to the PoW method of identifying the winner by hash
+calculation. On the other hand, it's hard to apply this scheme to our blockchain for several reasons: the number
+of Validators to be selected is non-deterministic and includes random behavior following a binomial distribution,
+the protocol complexity increases due to mutual recognition among the winning nodes, and it's not possible to find
+nodes that have been elected but have sabotaged their roles.
+
+We have considered a number of other consensus mechanisms, but we believe that the current choice is the closest
+realistic for role election and agreement algorithms for P2P distributed system. However, since Ostracon doesn't
+have a goal of experimental proofs or demonstrations for any particular research theory, we are ready to adopt better
+algorithms if they are proposed in the future.

docs/drafts/features/signature_aggregation.md

@@ -0,0 +1,47 @@
+# BLS Signature Aggregation
+
+## Overview
+
+Blockchains with a decentralized consensus mechanism need to collect a sufficient number of votes (signatures) each
+time a block is created. The more participants in a consensus, the more secure it becomes, but at the same time,
+the more signatures there are, the larger the block size becomes, and the longer it takes to verify, the worse
+the performance becomes. To solve this problem, Bitcoin (BIP340) and Ethereum 2.0 are working to improve performance
+by incorporating signature aggregation.
+
+The first paper on BLS signatures was published as a digital signature that could be implemented in a very small size,
+but the technique, called pairing, has led to several other interesting features, such as threshold signatures and
+blind signatures. Ostracon also aggregates the multiple signatures into a single one by BLS to improve performance
+by 1) reducing block size and 2) reducing the number of verifications.
+
+![BLS Signature Aggregation](bls_signature_aggregation.png)
+
+## Public Key Abstraction
+
+With the introduction of BLS signatures, Ostracon has been redesigned to allow signature keys with different schemes
+per node to be used on the same blockchain instance, which means that Ostracon participants can choose between fast
+and proven Ed25519 signatures and signature aggregation capable BLS signatures when setup their nodes. This flexibility
+gives us the flexibility to test/adopt better signature algorithms in the future, or to deal with vulnerabilities
+in the implementation if they are discovered.
+
+## Why is this an experimental status?
+
+In introducing BLS, we have unfortunately found that the BLS signature aggregation conflicts in several ways with
+the design of Tendermint, the base of Ostracon. A typical example is an elementary validation called Light Validation
+for light nodes. Even if a client doesn't have the public keys of all the nodes involved in the consensus, it can
+still consider a block to be correct if it successfully validates 2/3+1 of the total number of voters based on
+the BFT assumption. However, with BLS signatures, if even one of the public keys participating in the consensus is
+missing, all the aggregated signatures cannot be verified.
+
+In terms of performance, Ed25519 signatures are faster than BLS signatures for generating/verifying a single signature.
+We are carefully investigating where is the watershed point of the improvements where the block size reduction and
+the verification frequency reduction outweigh the slowness.
+
+| Algorithm         | Private Key | Public Key | Signature  | Sig Generation | Sig Verification  |
+|:------------------|------:|------:|-----:|--------:|--------:|
+| ECDSA (secp256k1) |   96B |   64B |  64B | 92μs    |   124μs |
+| Ed25519           |   64B |   32B |  64B | 49μs    |   130μs |
+| BLS12-381         |   32B |   96B |  48B | 233μs   | 1,149μs |
+Table: Space and time efficiency of signature algorithms. The message length for signature creation/verification is 1024 bytes.
+
+We have a plan to support BLS signatures going forward, but considering backward-incompatible fixes for these issues,
+this functionality currently has an experimental status.