NEP-364: Efficient signature verification host functions #364
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| --- | ||
| NEP: 364 | ||
| Title: Efficient signature verification and hashing precompile functions | ||
| Author: Blas Rodriguez Irizar <rodrigblas@gmail.com> | ||
| DiscussionsTo: https://github.com/nearprotocol/neps/pull/364 | ||
| Status: Draft | ||
| Type: Runtime Spec | ||
| Category: Contract | ||
| Created: 15-Jun-2022 | ||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| This NEP introduces the request of adding into the NEAR runtime a set of pre-compile | ||
| functions that can help IBC compatible light clients run on-chain. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Signature verification and hashing are ubiquitous operations in light clients, | ||
| especially in PoS consensus mechanisms. Based on the Polkadot's consensus mechanism | ||
| there will be a need for verification of ~200 signatures every minute | ||
| (Polkadot’s authority set is ~300 signers and it may be incresed in the future: https://polkadot.polkassembly.io/referendum/16). | ||
|
|
||
| Therefore, a mechanism to perform these operations in a cost-effective way | ||
| in terms of gas and speed would be highly beneficial to have. Currently, NEAR does not have any native signature verification toolbox. | ||
| This implies that a light client operating insid NEAR will have to import a library | ||
| compiled to WASM as mentioned in [Zulip](https://near.zulipchat.com/#narrow/stream/295302-general/topic/light_client). | ||
|
|
||
| Polkadot uses [three different cryptographic schemes](https://wiki.polkadot.network/docs/learn-keys) | ||
| for its keys/accounts, which also translates into different signatures types: | ||
|
|
||
| - The vanilla ed25519 implementation using Schnorr signatures. | ||
| - The Schnorrkel/Ristretto sr25519 variant using Schnorr signatures. | ||
| - ECDSA signatures on secp256k1 | ||
|
|
||
| ## Rationale and alternatives | ||
|
|
||
| Add three signature verification signatures functions into the runtime as host functions. | ||
|
|
||
| - ED25519 signature verification function | ||
| - SR25519 signature verification function | ||
| - ECDSA signature verification function | ||
|
|
||
| using `ed25519_dalek`, `schnorrkel` and `secp256k1` crates into NEAR runtime as pre-compiled functions. | ||
|
|
||
| Note: seems like `ecdsa_verify` is implemented, but called `erecover` in https://github.com/near/nearcore/blob/master/runtime/near-vm-logic/src/logic.rs#L1049 | ||
| Upon confirmation, we can proceed to remove it. | ||
|
|
||
| Benchmarks were run using a signature verifier smart contract on-chain importing the aforementioned functions from | ||
| widespread used crypto Rust crates. The biggest pitfall of that solution | ||
nagisa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| is performance and cost. Our [benchamarks](https://github.com/blasrodri/near-test) show the following results: | ||
|
|
||
| ```log | ||
| near call sitoula-test.testnet verify_ed25519 '{"signature_p1": [145,193,203,18,114,227,14,117,33,213,121,66,130,14,25,4,36,120,46,142,226,215,7,66,122,112,97,30,249,135,61,165], "signature_p2": [221,249,252,23,105,40,56,70,31,152,236,141,154,122,207,20,75,118,79,90,168,6,221,122,213,29,126,196,216,104,191,6], "msg": [107,97,106,100,108,102,107,106,97,108,107,102,106,97,107,108,102,106,100,107,108,97,100,106,102,107,108,106,97,100,115,107], "iterations": 10}' --accountId sitoula-test.testnet --gas 300000000000000 | ||
| # transaction id DZMuFHisupKW42w3giWxTRw5nhBviPu4YZLgKZ6cK4Uq | ||
|
|
||
| near call sitoula-test.testnet verify_schnorrkel '{"signature_p1": [106, 144, 17, 34, 142, 65, 191, 241, 233, 250, 132, 168, 204, 173, 122, 196, 118, 248, 159, 159, 254, 37, 153, 84, 248, 104, 206, 217, 168, 65, 12, 74], "signature_p2": [183, 134, 143, 30, 123, 61, 112, 153, 244, 109, 199, 195, 164, 0, 7, 55, 26, 199, 164, 219, 147, 217, 157, 239, 198, 108, 162, 246, 52, 49, 116, 132], "msg": [107,97,106,100,108,102,107,106,97,108,107,102,106,97,107,108,102,106,100,107,108,97,100,106,102,107,108,106,97,100,115,107], "iterations": 10}' --accountId sitoula-test.testnet --gas 300000000000000 | ||
| # transaction id DpWpK2jnyBRgZ7fScaBKULikJyvmMjDHx7AyWMrJL5VB | ||
|
|
||
| near call sitoula-test.testnet verify_ecdsa '{"signature_p1": [231, 117, 17, 89, 49, 142, 111, 201, 161, 107, 167, 147, 215, 167, 196, 226, 200, 176, 184, 62, 196, 240, 210, 137, 77, 198, 90, 97, 201, 212, 96, 229], "signature_p2": [1, 31, 7, 121, 178, 247, 150, 131, 108, 250, 173, 71, 100, 192, 83, 64, 145, 85, 254, 69, 176, 7, 114, 89, 64, 205, 30, 243, 193, 78, 142, 27], "msg": [107,97,106,100,108,102,107,106,97,108,107,102,106,97,107,108,102,106,100,107,108,97,100,106,102,107,108,106,97,100,115,107], "iterations": 10}' --accountId sitoula-test.testnet --gas 300000000000000 | ||
| # transaction id AKFwCuQ8g2a8t9SX7vcve348BvwoU3M42MSef7gy1kPA | ||
| ``` | ||
|
|
||
| With `iterations = 130` **all these calls return ExecutionError**: `'Exceeded the maximum amount of gas allowed to burn per contract.'` | ||
| With iterations = 50 these are the results: | ||
|
Comment on lines
+49
to
+50
There was a problem hiding this comment. Is one iteration one signature verification? There was a problem hiding this comment.
I've added a reference in the motivation section. Would you like it to be more specific, being included in the comment of each host function? |
||
|
|
||
| ``` | ||
| ed25518: tx id 6DcJYfkp9fGxDGtQLZ2m6PEDBwKHXpk7Lf5VgDYLi9vB (299 Tgas) | ||
| schnorrkel: tx id ACYJh7YC4pQM8fu7DsjmmkSb3WSvikfQkVyej3htiuDQ (304 Tgas) | ||
| ecdsa: tx id 2FhtFEbakuwQyHWcyeRoPkCDgXz4erTYSD75QkxDTb2e (185 Tgas) | ||
| ``` | ||
|
|
||
| - Performance in wall clock time when you compile the signature validation library directly from rust to native. | ||
| Here are the results on a AMD Ryzen 9 5900X 12-Core Processor machine: | ||
|
|
||
| ``` | ||
| # 10k signature verifications | ||
| ed25519: took 387ms | ||
| ecdsa: took 324ms | ||
| shnorrkel: took 411ms | ||
| ``` | ||
|
|
||
| - Performance in wall clock time when you compile the library into wasm first and then use the single-pass compiler in Wasmer 1 to then compile to native. | ||
|
|
||
| ``` | ||
| ed25519: took 9926ms | ||
| ecdsa: took 6725ms | ||
| shnorrkel: took 10220ms | ||
| ``` | ||
|
|
||
| As an extra datapoint, when passing `--enable-simd` instead of `--singlepass` | ||
|
|
||
| ``` | ||
| ed25519: took 3085ms | ||
| ecdsa: took 1895ms | ||
| shnorrkel: took 3179ms | ||
| ``` | ||
|
|
||
| Steps to reproduce: | ||
| commit: `31cf97fb2e155d238308f062c4b92bae716ac19f` in `https://github.com/blasrodri/near-test` | ||
|
|
||
| ```sh | ||
| # wasi singlepass | ||
| cargo wasi build --bin benches --release | ||
| wasmer compile --singlepass ./target/wasm32-wasi/release/benches.wasi.wasm -o benches_singlepass | ||
| wasmer run ./benches_singlepass | ||
| ``` | ||
|
|
||
| ```sh | ||
| # rust native | ||
| cargo run --bin benches --release | ||
| ``` | ||
|
|
||
| Overall: the difference between the two versions (native vs wasi + singlepass are) | ||
|
|
||
| ``` | ||
| ed25519: 25.64x slower | ||
| ecdsa: 20.75x slower | ||
| shnorrkel: 24.86x slower | ||
| ``` | ||
|
There was a problem hiding this comment. can you also mention is the expected improvement after this NEP ? How many verifications are you able to fit in a 300 TGas transaction ? |
||
|
|
||
| ### What is the impact of not doing this? | ||
|
|
||
| Costs of running IBC compatible trustless bridges would be very high. Plus, the fact that signature verification | ||
| is such an expensive operation will force the contract to split the process of batch verification of signatures | ||
| into multiple transactions. | ||
|
|
||
| ### Why is this design the best in the space of possible designs? | ||
|
|
||
| Adding exisiting proved and vetted crypto crates into the runtime is a safe workaround. It will boost performance | ||
| between 20-25x according to our benchmarks. This will both reduce operation costs significantly and will also | ||
| enable the contract to verify all the signatures in one transaction, which will simplify the contract design. | ||
|
There was a problem hiding this comment. If Polkadot uses 3 different types of keys - are you still able to verify all the signatures in one transaction ? (for example in a scenario, where all the polkadot validators are using sr25519 ? ) |
||
|
|
||
| ### What other designs have been considered and what is the rationale for not choosing them? | ||
|
|
||
| Adding exisiting proved and vetted crypto crates into the runtime is a safe workaround. It will boost performance | ||
| between 20-25x according to our benchmarks. This will both reduce operation costs significantly and will also | ||
| enable the contract to verify all the signatures in one transaction, which will simplify the contract design. | ||
nagisa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Specification | ||
nagisa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| This NEP aims to introduce the following host functions: | ||
|
|
||
| ```rust | ||
| pub fn ed25519_verify(sig: &ed25519::Signature, msg: &[u8], pub_key: &ed25519::Public) -> bool | ||
| pub fn sr25519_verify(sig: &sr25519::Signature, msg: &[u8], pub_key: &sr25519::Public) -> bool | ||
| pub fn ecdsa_verify(sig: &ecdsa::Signature, msg: &[u8], pub_key: &ecdsa::Public) -> bool | ||
| ``` | ||
blasrodri marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Once this NEP is approved and integrated, these four functions will be available in the `near_sdk` crate in the | ||
blasrodri marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| `env` module. | ||
|
|
||
| ## Security Implications (Optional) | ||
|
|
||
| We have chosen these crates because they are the ones already integrated in the [sp_io](https://docs.rs/sp-io/latest/sp_io/) | ||
| crate heavily used in the [substrate](https://github.com/paritytech/substrate) repository. | ||
|
|
||
| ## Unresolved Issues (Optional) | ||
|
|
||
| - What parts of the design do you expect to resolve through the NEP process before this gets merged? | ||
| Both the function signatures and crates are up for discussion. | ||
|
There was a problem hiding this comment. I suggest that we use the library that nearcore already uses for ed25519 signatures. This would avoid introducing new dependencies and ensure that NEAR signatures themselves can be 100% verified in smart contracts There was a problem hiding this comment. As far as I can see the proposal is the same as the one being used today |
||
|
|
||
| ## Future possibilities | ||
|
|
||
| I currently do not envision any extension in this regard. | ||
|
|
||
| ## Copyright | ||
|
|
||
| [copyright]: #copyright | ||
|
|
||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please set the correct number
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unrelated to this specific PR, but I see a process issue (so, cc @frol): our process for NEPs should not require modification of shared files (like readme.md), as that is very prone to conflicts.