NEP-364: Efficient signature verification host functions

neps/nep-0364.md

@@ -0,0 +1,199 @@
+---
+NEP: 364
+Title: Efficient signature verification and hashing precompile functions
+Author: Blas Rodriguez Irizar <rodrigblas@gmail.com>
+DiscussionsTo: https://github.com/nearprotocol/neps/pull/364
+Status: Draft
+Type: Runtime Spec
+Category: Contract
+Created: 15-Jun-2022
+---
+
+## Summary
+
+This NEP introduces the request of adding into the NEAR runtime a pre-compiled
+function used to verify signatures that can help IBC compatible light clients run on-chain.
+
+## Motivation
+
+Signature verification and hashing are ubiquitous operations in light clients,
+especially in PoS consensus mechanisms. Based on Polkadot's consensus mechanism
+there will be a need for verification of ~200 signatures every minute
+(Polkadot’s authority set is ~300 signers and it may be increased in the future: https://polkadot.polkassembly.io/referendum/16).
+
+Therefore, a mechanism to perform these operations cost-effectively in terms
+of gas and speed would be highly beneficial to have. Currently, NEAR does not have any native signature verification toolbox.
+This implies that a light client operating inside NEAR will have to import a library
+compiled to WASM as mentioned in [Zulip](https://near.zulipchat.com/#narrow/stream/295302-general/topic/light_client).
+
+Polkadot uses [three different cryptographic schemes](https://wiki.polkadot.network/docs/learn-keys)
+for its keys/accounts, which also translates into different signature types. However, for this NEP the focus is on:
+
+- The vanilla ed25519 implementation uses Schnorr signatures.
+
+## Rationale and alternatives
+
+Add a signature verification signatures function into the runtime as host functions.
+
+- ED25519 signature verification function using `ed25519_dalek` crates into NEAR runtime as pre-compiled functions.
+
+Benchmarks were run using a signature verifier smart contract on-chain importing the aforementioned functions from
+widespread used crypto Rust crates. The biggest pitfall of these functions running wasm code instead of native
+is performance and gas cost. Our [benchmarks](https://github.com/blasrodri/near-test) show the following results:
+
+```log
+near call sitoula-test.testnet verify_ed25519 '{"signature_p1": [145,193,203,18,114,227,14,117,33,213,121,66,130,14,25,4,36,120,46,142,226,215,7,66,122,112,97,30,249,135,61,165], "signature_p2": [221,249,252,23,105,40,56,70,31,152,236,141,154,122,207,20,75,118,79,90,168,6,221,122,213,29,126,196,216,104,191,6], "msg": [107,97,106,100,108,102,107,106,97,108,107,102,106,97,107,108,102,106,100,107,108,97,100,106,102,107,108,106,97,100,115,107], "iterations": 10}' --accountId sitoula-test.testnet --gas 300000000000000
+# transaction id DZMuFHisupKW42w3giWxTRw5nhBviPu4YZLgKZ6cK4Uq
+```
+
+With `iterations = 130` **all these calls return ExecutionError**: `'Exceeded the maximum amount of gas allowed to burn per contract.'`
+With iterations = 50 these are the results:
+
+```
+ed25519: tx id 6DcJYfkp9fGxDGtQLZ2m6PEDBwKHXpk7Lf5VgDYLi9vB (299 Tgas)
+```
+
+- Performance in wall clock time when you compile the signature validation library directly from rust to native.
+  Here are the results on an AMD Ryzen 9 5900X 12-Core Processor machine:
+
+```
+# 10k signature verifications
+ed25519: took 387ms
+```
+
+- Performance in wall clock time when you compile the library into wasm first and then use the single-pass compiler in Wasmer 1 to then compile to native.
+
+```
+ed25519: took 9926ms
+```
+
+As an extra data point, when passing `--enable-simd` instead of `--singlepass`
+
+```
+ed25519: took 3085ms
+```
+
+Steps to reproduce:
+commit: `31cf97fb2e155d238308f062c4b92bae716ac19f` in `https://github.com/blasrodri/near-test`
+
+```sh
+# wasi singlepass
+cargo wasi build --bin benches --release
+wasmer compile --singlepass ./target/wasm32-wasi/release/benches.wasi.wasm -o benches_singlepass
+wasmer run ./benches_singlepass
+```
+
+```sh
+# rust native
+cargo run --bin benches --release
+```
+
+Overall: the difference between the two versions (native vs wasi + singlepass is)
+
+```
+ed25519: 25.64x slower
+```
+
+### What is the impact of not doing this?
+
+Costs of running IBC-compatible trustless bridges would be very high. Plus, the fact that signature verification
+is such an expensive operation will force the contract to split the process of batch verification of signatures
+into multiple transactions.
+
+### Why is this design the best in the space of possible designs?
+
+Adding existing proved and vetted crypto crates into the runtime is a safe workaround. It will boost performance
+between 20-25x according to our benchmarks. This will both reduce operating costs significantly and will also
+enable the contract to verify all the signatures in one transaction, which will simplify the contract design.
+
+### What other designs have been considered and what is the rationale for not choosing them?
+
+One possible alternative would be to improve the runtime implementation so that it can compile WASM code to a sufficiently
+fast machine code. Even when it may not be as fast as LLVM native produced code it could still be acceptable for
+these types of use cases (CPU intensive functions) and will certainly avoid the need of adding host functions.
+The effort of adding such a feature will be significantly higher than adding these host functions one by one.
+But on the other side, it will decrease the need of including more host functions in the future.
+
+Another alternative is to deal with the high cost of computing/verifying these signatures in some other manner.
+Decreasing the overall cost of gas and increasing the limits of gas available to attach to the contract could be a possibility.
+Introducing such modification for some contracts, and not for some others can be rather arbitrary
+and not straightforward in the implementation, but an alternative nevertheless.
+
+## Specification
+
+This NEP aims to introduce the following host function:
+
+```rust
+/// Ed25519 is a public-key signature system with several attractive features
+///
+/// Proof of Stake Validator sets can contain different signature schemes.
+/// Ed25519 is one of the most used ones across blockchains, and hence it's importance to be added.
+/// For further reference, visit: https://ed25519.cr.yp.to
+///
+/// Verify an ED25519 signature given a message and a public key.
+/// # Returns
+/// - 1 meaning the boolean expression true to encode that the signature was properly verified
+/// - 0 meaning the boolean expression false to encode that the signature failed to be verified
+///
+/// # Cost
+///
+/// Each input can either be in memory or in a register. Set the length of the input to `u64::MAX`
+/// to declare that the input is a register number and not a pointer.
+/// Each input has a gas cost input_cost(num_bytes) that depends on whether it is from memory
+/// or from a register. It is either read_memory_base + num_bytes * read_memory_byte in the
+/// former case or read_register_base + num_bytes * read_register_byte in the latter. This function
+/// is labeled as `input_cost` below.
+///
+/// `input_cost(num_bytes_signature + num_bytes_message, num_bytes_public_key) +
+///  ed25519_verify_base + ed25519_verify_byte * (num_bytes_signature + num_bytes_message)`
+///
+/// # Errors
+///
+/// The signature size is fixed and equal to SIGNATURE_LENGTH (64 bytes). In case the length of signature input is not equal to
+/// SIGNATURE_LENGTH, then the function returns HostError::Ed25519VerifyInvalidInput with the message "invalid signature length".
+/// 
+/// A similar case occurs with the public key. Its size is known and its equal to PUBLIC_KEY_LENGTH (32 bytes). In case the
+/// input length provided for the publc key doesn't match PUBLIC_KEY_LENGTH, the function will return the following error:
+/// HostError::Ed25519VerifyInvalidInput with the message "invalid public key length".
+pub fn ed25519_verify(
+    &mut self,
+    sig_len: u64,
+    sig_ptr: u64,
+    msg_len: u64,
+    msg_ptr: u64,
+    pub_key_len: u64,
+    pub_key_ptr: u64,
+) -> Result<u64>;
+```
+
+And a `rust-sdk` possible implementation could look like this:
+
+```rs
+
+pub fn ed25519_verify(sig: &ed25519::Signature, msg: &[u8], pub_key: &ed25519::Public) -> bool;
+
+```
+The current implementation is imported from the crate `ed25519-dalek`, version 1. It uses no feature flags
+except from the default one. This is the exact same setup used in the crate `near-crypto`.
+
+Once this NEP is approved and integrated, these functions will be available in the `near_sdk` crate in the
+`env` module.
+
+## Security Implications (Optional)
+
+We have chosen this crate because it is already integrated into `nearcore`.
+
+## Unresolved Issues (Optional)
+
+- What parts of the design do you expect to resolve through the NEP process before this gets merged?
+  Both the function signatures and crates are up for discussion.
+
+## Future possibilities
+
+I currently do not envision any extension in this regard.
+
+## Copyright
+
+[copyright]: #copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).