Uniswap benchmark on Aurora (NEAR gas usage)

[birchmd]

How to run benchmark:

git clone https://github.com/birchmd/aurora-engine.git
cd aurora-engine
git checkout uniswap-benchmark
make testnet
cargo test --features=testnet-test --lib uniswap -- --nocapture

Result (adding liquidity transaction -- 593k EVM gas):

------------------------------
Total gas: 216675334602795
Host gas: 29371572525861 [13% total]
Action gas: 0 [0% total]
Wasm execution: 187303762076934 [86% total]
------ Host functions --------
base -> 183484300923 [0% total, 0% host]
contract_compile_base -> 35445963 [0% total, 0% host]
contract_compile_bytes -> 191852361000 [0% total, 0% host]
read_memory_base -> 934331025600 [0% total, 3% host]
read_memory_byte -> 53256675330 [0% total, 0% host]
write_memory_base -> 521505844146 [0% total, 1% host]
write_memory_byte -> 788177903640 [0% total, 2% host]
read_register_base -> 470709889782 [0% total, 1% host]
read_register_byte -> 28527193908 [0% total, 0% host]
write_register_base -> 618952856976 [0% total, 2% host]
write_register_byte -> 1567027490184 [0% total, 5% host]
keccak256_base -> 11758982550 [0% total, 0% host]
keccak256_byte -> 10005534930 [0% total, 0% host]
ecrecover_base -> 3365369625000 [1% total, 11% host]
storage_write_base -> 2760459648000 [1% total, 9% host]
storage_write_key_byte -> 131027649753 [0% total, 0% host]
storage_write_value_byte -> 43922251224 [0% total, 0% host]
storage_write_evicted_byte -> 23895276408 [0% total, 0% host]
storage_read_base -> 14652779895000 [6% total, 49% host]
storage_read_key_byte -> 263591771028 [0% total, 0% host]
storage_read_value_byte -> 2305382402340 [1% total, 7% host]
storage_remove_base -> 427784244000 [0% total, 1% host]
storage_remove_key_byte -> 17734258176 [0% total, 0% host]
------ Actions --------
------------------------------

Result (swap transaction -- 135k EVM gas):

------------------------------
Total gas: 143362664464938
Host gas: 17157351033843 [11% total]
Action gas: 0 [0% total]
Wasm execution: 126205313431095 [88% total]
------ Host functions --------
base -> 107231084955 [0% total, 0% host]
contract_compile_base -> 35445963 [0% total, 0% host]
contract_compile_bytes -> 191852361000 [0% total, 1% host]
read_memory_base -> 480214828800 [0% total, 2% host]
read_memory_byte -> 23537853936 [0% total, 0% host]
write_memory_base -> 336455383320 [0% total, 1% host]
write_memory_byte -> 372808121184 [0% total, 2% host]
read_register_base -> 304576987506 [0% total, 1% host]
read_register_byte -> 13496686032 [0% total, 0% host]
write_register_base -> 429828372900 [0% total, 2% host]
write_register_byte -> 940484124216 [0% total, 5% host]
keccak256_base -> 11758982550 [0% total, 0% host]
keccak256_byte -> 7944308850 [0% total, 0% host]
ecrecover_base -> 3365369625000 [2% total, 19% host]
storage_write_base -> 1540721664000 [1% total, 8% host]
storage_write_key_byte -> 53355530319 [0% total, 0% host]
storage_write_value_byte -> 25062979512 [0% total, 0% host]
storage_write_evicted_byte -> 23895276408 [0% total, 0% host]
storage_read_base -> 7439103639000 [5% total, 43% host]
storage_read_key_byte -> 108457675632 [0% total, 0% host]
storage_read_value_byte -> 1381160102760 [0% total, 8% host]
------ Actions --------
------------------------------

[birchmd]

Update: wall-clock measurements

As of this commit there is a separate benchmark which measures wall-clock time. Inspired by the measurements done by @matklad, it measures both with and without gas injection into the wasm (taking advantage of the fact that injection does not happen if the regular_op_cost is zero) and using both wasmer0 and wasmer1 VMs.

How to run benchmark:

cargo test --release --features=testnet-test --lib uniswap -- --ignored --nocapture

Result (adding liquidity transaction -- 593k EVM gas):

uniswap-wasmer0/add_liquidity         time:   [289.07 ms 289.24 ms 289.44 ms]
uniswap-wasmer1/add_liquidity         time:   [59.763 ms 59.797 ms 59.832 ms]
uniswap-wasmer0-no-gas/add_liquidity  time:   [14.967 ms 15.033 ms 15.101 ms]
uniswap-wasmer1-no-gas/add_liquidity  time:   [13.837 ms 13.849 ms 13.863 ms]

Result (swap transaction -- 135k EVM gas):

uniswap-wasmer0/swap         time:   [185.35 ms 185.42 ms 185.50 ms]
uniswap-wasmer1/swap         time:   [38.596 ms 38.636 ms 38.693 ms]
uniswap-wasmer0-no-gas/swap  time:   [10.809 ms 10.844 ms 10.880 ms]
uniswap-wasmer1-no-gas/swap  time:   [9.6091 ms 9.6212 ms 9.6394 ms]

Conclusions:

• In wasmer0, not injecting the gas instructions into the wasm has nearly a 20x improvement on wall-clock time.
• Wasmer1 performs signficantly better than wasmer0 when gas instructions are injected
• Wasmer0 and Wasmer1 perform similarly when no gas instructions are injected