Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(levm): pectra-devnet6 eftests #1877

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

fborello-lambda
Copy link
Contributor

@fborello-lambda fborello-lambda commented Feb 5, 2025

Motivation

The latest test for pectra has been released. Pectra Devnet 6

Description

Some tests fail.
Especifically for EIP7702:

  • tests/prague/eip7702_set_code_tx/test_set_code_txs_2.py::test_eoa_init_as_pointer[fork_Prague-state_test]
cd crates/vm/levm
make run-evm-ef-tests flags="-- --specific-tests tests/prague/eip7702_set_code_tx/test_set_code_txs_2.py::test_eoa_init_as_pointer[fork_Prague-state_test]"

Especifically for EIP7623:

cd crates/vm/levm
make run-evm-ef-tests flags="--tests refunds"

@fborello-lambda fborello-lambda added the levm Lambda EVM implementation label Feb 5, 2025
@fborello-lambda fborello-lambda self-assigned this Feb 5, 2025
Copy link

github-actions bot commented Feb 5, 2025

| File                                                                              | Lines | Diff |
+-----------------------------------------------------------------------------------+-------+------+
| /home/runner/work/ethrex/ethrex/crates/vm/levm/src/opcode_handlers/environment.rs | 359   | -16  |
+-----------------------------------------------------------------------------------+-------+------+

Total lines added: 0
Total lines removed: 16
Total lines changed: 16

Copy link

github-actions bot commented Feb 5, 2025

Benchmark Results Comparison

PR Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 240.4 ± 0.6 239.5 241.2 1.00
levm_Factorial 905.8 ± 10.4 891.8 923.3 3.77 ± 0.04

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.508 ± 0.098 1.369 1.636 1.00
levm_FactorialRecursive 15.689 ± 0.125 15.506 15.864 10.40 ± 0.68

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 214.3 ± 1.2 213.0 216.5 1.00
levm_Fibonacci 903.2 ± 11.1 885.4 921.9 4.21 ± 0.06

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.8 ± 0.0 8.8 8.9 1.00
levm_ManyHashes 18.2 ± 0.3 18.0 19.0 2.07 ± 0.03

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.268 ± 0.010 3.254 3.291 1.00
levm_BubbleSort 6.101 ± 0.035 6.056 6.170 1.87 ± 0.01

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 251.3 ± 0.9 250.1 252.6 1.00
levm_ERC20Transfer 541.1 ± 3.3 537.3 548.4 2.15 ± 0.02

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 144.0 ± 0.9 143.0 145.5 1.00
levm_ERC20Mint 357.3 ± 10.2 350.5 385.4 2.48 ± 0.07

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.042 ± 0.005 1.035 1.049 1.00
levm_ERC20Approval 2.034 ± 0.032 2.004 2.120 1.95 ± 0.03

Main Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 242.1 ± 1.8 239.9 246.4 1.00
levm_Factorial 900.7 ± 13.1 888.5 925.1 3.72 ± 0.06

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.463 ± 0.075 1.372 1.596 1.00
levm_FactorialRecursive 15.660 ± 0.126 15.477 15.790 10.70 ± 0.56

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 213.9 ± 0.7 212.5 214.9 1.00
levm_Fibonacci 900.0 ± 8.3 887.5 910.0 4.21 ± 0.04

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.9 ± 0.1 8.8 9.3 1.00
levm_ManyHashes 18.2 ± 0.3 17.8 18.6 2.04 ± 0.04

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.247 ± 0.031 3.224 3.331 1.00
levm_BubbleSort 6.088 ± 0.034 6.053 6.147 1.87 ± 0.02

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 255.8 ± 1.1 254.3 258.5 1.00
levm_ERC20Transfer 536.8 ± 3.2 531.5 540.9 2.10 ± 0.02

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 147.2 ± 1.0 145.4 148.6 1.00
levm_ERC20Mint 350.0 ± 2.4 347.6 355.6 2.38 ± 0.02

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.064 ± 0.010 1.052 1.083 1.00
levm_ERC20Approval 2.014 ± 0.017 1.992 2.047 1.89 ± 0.02

Copy link

EF Tests Comparison

Test Name MAIN PR DIFF
Summary: 16315/16773 (97.27%) 19626/20129 (97.50%) ⬆️ 3311
Prague: 2373/2373 (100.00%) 2570/2574 (99.84%) ⬇️️ 197
Cancun: 3579/3579 (100.00%) 4983/4983 (100.00%) ➖️ 1404
Shanghai: 221/221 (100.00%) 614/614 (100.00%) ➖️ 393
Paris: 62/62 (100.00%) 285/285 (100.00%) ➖️ 223
London: 39/39 (100.00%) 247/247 (100.00%) ➖️ 208
Berlin: 35/35 (100.00%) 205/239 (85.77%) ⬇️️ 170
Istanbul: 35/35 (100.00%) 229/230 (99.57%) ⬇️️ 194
Petersburg: 2363/2399 (98.50%) 2502/2541 (98.47%) ⬇️️ 139
Constantinople: 2323/2405 (96.59%) 2323/2405 (96.59%) ➖️
Byzantium: 2294/2330 (98.45%) 2433/2472 (98.42%) ⬇️️ 139
SpuriousDragon: 579/579 (100.00%) 579/579 (100.00%) ➖️
Tangerine: 563/650 (86.62%) 563/650 (86.62%) ➖️
Homestead: 1208/1324 (91.24%) 1331/1447 (91.98%) ⬆️ 123
Frontier: 641/742 (86.39%) 762/863 (88.30%) ⬆️ 121

@lambdaclass lambdaclass deleted a comment from github-actions bot Feb 20, 2025
@fborello-lambda fborello-lambda marked this pull request as ready for review February 20, 2025 19:50
@fborello-lambda fborello-lambda requested a review from a team as a code owner February 20, 2025 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
levm Lambda EVM implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant