Skip to content

Commit

Permalink
[AA] Support for multiple entryPoints (#3781)
Browse files Browse the repository at this point in the history 
* 🔥 Changing vars in Plugin file (does not compile)

* Support added for multiple entry points-1

* Updating Fast Sync config files (#3780)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* Update and fix ethereum tests (#3778)

* updated tests

* update tests reference to the most recent commit

* update tests reference

* fix loaders

* delete stale tests

* update tests

* fix BerlinToLondon tests

* ignore 12 transactions tests for now (passing hive tests, failing here)

* update TransactionTests runner

* disable difficulty tests with broken loader

* move VM tests from Blockchain.Test project to VM.Test project

* clean

* fix workflow and add missing tests

* remove duplicated tests

* fix MetaTest

* add MetaTest for VM.Test

* cosmetics

* suggested changes

* use dotnet-dump only

* Update Dockerfile.dev

* Update Dockerfile.dev

* Update Dockerfile.dev

* Update Dockerfile.dev

* Update Dockerfile.dev

* Update Dockerfile.dev

* Updating Fast Sync config files (#3784)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* Updating Fast Sync config files (#3787)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* Updating Fast Sync config files (#3788)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* Full pruning (#3096)

* Add HighestPersistedState to BlockTree

* RocksDbSettings cloning

* MemDbFactoryAdapter to use IRocksDbFactory

* Pruning Triggers

* Add StateDbDuplicateWrites metrics

* Add FullPruningDb

* Add FullPruner and CopyTreeVisitor

* CopyTreeVisitor add logs

* CopyTreeVisitor add tests

* Rename RocksDbSizePruningTrigger to PathSizePruningTrigger

* add tests for PathSizePruningTrigger

* Move test projects to c# 9

* Add DiskFreeSpacePruningTrigger Tests

* Add Build.A.Trie and TrieBuilder

* Move PruningConfig add PruningMode

* FullPruningInnerDbFactory to correctly read db index

* Add RocksDbSettings clone test

* Change FullPruningDb, add tests for it

* Rename MemDbFactoryToRocksDbAdapter

* Change StandardDbInitializer to support full pruning

* Add FullPruner tests

* Add TrieStoreBondryWatcher to encapsulate and reuse connection between TrieStore and BlockTree

* Fix HighestPersistedState null serialization

* Fix DbOnTheRocks Clear by delete (needed dispose first) and special prefix for State db

* Add FullPruningDiskTest

* Add PruningConfig.FullPruningThresholdMb and PruningConfig.FullPruningTrigger

* Add IFullPruningDb.GetPath

* Initialize FullPruner

* Better metrics

* BlockTree highest persisted state fix

* fixes

* Duplicate batches in pruning, start full pruning only between blocks, fix tests

* Fix FullPruningDb canellation

* Fix indexing db in PathSizePruningTrigger

* Fix FullPruner, use head.StateRoot

* fix delete database

* fix tests

* fix tests

* Change config a bit

* Make CopyTreeVisitor parallel

* Move to VisitingOptions

* add FullPruningMaxDegreeOfParallelism config option

* Remove SupportsParallelVisits

* changes

* Add full pruned configs

* Fix issues with TrieStore with both archive and mem pruning modes.

Add Full pruning RPC.

* Refactor + comments

* FIx initialize pruning trigger when only Manual pruning is enabled

* Move parallelism to be based on Semaphore

* PruningTriggerPersistenceStrategy experiment

* Add description how Full Pruning works

* Fix issues with hybrid pruning

* fix resolving storage root

* refactor merge

* Add pruning delay to make sure we won't break ssds

* optimize ShouldPersist

* Add TrieStats Levels

* Make TrieStats collecting parallel

* Make Trie Stats collecting in the background

* Hide PruningConfig.Enabled from docs

* Log Exception when collecting trie stats

* Always register FullPruningDb if someone disables it in the future

* small refactor

* Set VisitContext.MaxDegreeOfParallelism default to 1

* Remove IDb.Innermost

* Remove non-default module types

* fix tests

* review comments

Co-authored-by: Jorge Mederos Alvarado <jmederosalvarado@gmail.com>
Co-authored-by: DCeleda <dceleda@hotmail.com>

* Updating Fast Sync config files (#3792)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* Accept underpaid local txs in FeeTooLowFilter (#3794)

* always accept local txs in FeeTooLowFilter

* add test

* fix typo

* cosmetic

* add own txs length check to test

* Update rocks db pointer (#3795)

* Clone BlockHeader before executing RPC (#3800)

* Sepolia - DifficultyBombDisabled (#3789)

* Fix sepolia

* more tests

* cosmetic

* fix test

* move disabled to sepolia chainspec

* review fix

* review fix v2

* Updating Fast Sync config files (#3803)

Co-authored-by: matilote <matilote@users.noreply.github.com>

* [AA] Only allow max 10 User Operation per sender, Implement op fee replacement (#3799)

* Support added for multiple entry points-1

* Support added for multiple entry points-2

* AccountAbstractionRpcModule modified

* Some Tests fixed

* wrote p2p logic

* Made changes to other functions wrt the p2p commit

* Errors Fixed

* Prep for adding tests for checking support for multiple entry points

* Tests updated

* fixed deployment

* Added Tests-2

* merge conflict WIP

* Tests added for AaPeerManager

* fixes

* fixes

* New tests added

* Finalized the tests

* fixing user op tx source

* fix stateprovider issue

* fixed jorge notes

* update configs

* update rocksdb

* Rollback rocksdb update (#3850)

* add accountabstraction plugins to default plugins

* fixed null reference bug

Co-authored-by: Parinay Chauhan <parinayc20@iitk.ac.in>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: matilote <matilote@users.noreply.github.com>
Co-authored-by: Marcin Sobczak <77129288+marcindsobczak@users.noreply.github.com>
Co-authored-by: Mateusz Jędrzejewski <33068017+matilote@users.noreply.github.com>
Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>
Co-authored-by: Jorge Mederos Alvarado <jmederosalvarado@gmail.com>
Co-authored-by: DCeleda <dceleda@hotmail.com>
Co-authored-by: Marek Moraczyński <marekm2504@gmail.com>
  • Loading branch information
@kristofgazso @parinayc20
@github-actions @matilote @marcindsobczak @LukaszRozmej @jmederosalvarado @dceleda @MarekM25
10 people authored Mar 11, 2022
1 parent cbc9e1e commit 114d0cc
Show file tree
Hide file tree
Showing 52 changed files with 3,104 additions and 450 deletions.
148 changes: 148 additions & 0 deletions src/Nethermind/Nethermind.AccountAbstraction.Test/AccountAbstractionPeerManagerTests.cs
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Antlr4.Runtime.Misc;
using FluentAssertions;
using Nethermind.AccountAbstraction.Broadcaster;
using Nethermind.AccountAbstraction.Data;
using Nethermind.AccountAbstraction.Executor;
using Nethermind.AccountAbstraction.Network;
using Nethermind.AccountAbstraction.Source;
using Nethermind.Blockchain;
using Nethermind.Blockchain.Filters;
using Nethermind.Blockchain.Filters.Topics;
using Nethermind.Blockchain.Find;
using Nethermind.Blockchain.Receipts;
using Nethermind.Consensus;
using Nethermind.Core;
using Nethermind.Core.Crypto;
using Nethermind.Core.Extensions;
using Nethermind.Logging;
using Nethermind.State;
using NSubstitute;
using NUnit.Framework;
using Nethermind.Core.Test.Builders;
using Nethermind.Crypto;
using Nethermind.Int256;
using Nethermind.JsonRpc;

namespace Nethermind.AccountAbstraction.Test
{
[TestFixture]
public class AccountAbstractionPeerManagerTests
{
private IDictionary<Address, UserOperationPool> _userOperationPools = new Dictionary<Address, UserOperationPool>();
private IUserOperationSimulator _simulator = Substitute.For<IUserOperationSimulator>();
private IBlockTree _blockTree = Substitute.For<IBlockTree>();
private ILogger _logger = Substitute.For<ILogger>();
private ILogFinder _logFinder = Substitute.For<ILogFinder>();
private IStateProvider _stateProvider = Substitute.For<IStateProvider>();
private readonly ISigner _signer = Substitute.For<ISigner>();
private readonly string[] _entryPointContractAddress = {"0x8595dd9e0438640b5e1254f9df579ac12a86865f", "0x96cc609c8f5458fb8a7da4d94b678e38ebf3d04e"};
private static Address _notAnAddress = new("0x373f2D08b1C195fF08B9AbEdE3C78575FAAC2aCf");

[Test]
public void should_add_peers()
{
UserOperationBroadcaster _broadcaster = new UserOperationBroadcaster(_logger);
GenerateMultiplePools();
AccountAbstractionPeerManager _peerManager =
new AccountAbstractionPeerManager(_userOperationPools, _broadcaster, _logger);
IList<IUserOperationPoolPeer> peers = GetPeers();
foreach (IUserOperationPoolPeer peer in peers)
{
_peerManager.AddPeer(peer);
}
}

[Test]
public void should_delete_peers()
{
UserOperationBroadcaster _broadcaster = new UserOperationBroadcaster(_logger);
GenerateMultiplePools();
AccountAbstractionPeerManager _peerManager =
new AccountAbstractionPeerManager(_userOperationPools, _broadcaster, _logger);
IList<IUserOperationPoolPeer> peers = GetPeers();

foreach (IUserOperationPoolPeer peer in peers)
{
_peerManager.AddPeer(peer);
}

foreach (IUserOperationPoolPeer peer in peers)
{
_peerManager.RemovePeer(peer.Id);
}
}

private IList<IUserOperationPoolPeer> GetPeers(int limit = 100)
{
IList<IUserOperationPoolPeer> peers = new List<IUserOperationPoolPeer>();
for (int i = 0; i < limit; i++)
{
PrivateKey privateKey = new((i + 1).ToString("x64"));
peers.Add(GetPeer(privateKey.PublicKey));
}

return peers;
}
private IUserOperationPoolPeer GetPeer(PublicKey publicKey)
{
IUserOperationPoolPeer peer = Substitute.For<IUserOperationPoolPeer>();
peer.Id.Returns(publicKey);

return peer;
}

private void GenerateMultiplePools()
{
for (int i = 0; i < _entryPointContractAddress.Length; i++)
{
Address entryPoint = new Address(_entryPointContractAddress[i]);
_userOperationPools[entryPoint] = GenerateUserOperationPool(entryPoint, 100);
}
}

private UserOperationPool GenerateUserOperationPool(Address entryPoint, int capacity = 10, int perSenderCapacity = 10 )
{
IAccountAbstractionConfig config = Substitute.For<IAccountAbstractionConfig>();
// config.EntryPointContractAddresses.Returns(_entryPointContractAddress);
config.UserOperationPoolSize.Returns(capacity);
config.MaximumUserOperationPerSender.Returns(perSenderCapacity);

UserOperationSortedPool userOperationSortedPool =
new(capacity, CompareUserOperationsByDecreasingGasPrice.Default, LimboLogs.Instance, config.MaximumUserOperationPerSender);

_stateProvider.GetBalance(Arg.Any<Address>()).Returns(1.Ether());
_stateProvider.AccountExists(Arg.Any<Address>()).Returns(true);
_stateProvider.IsContract(Arg.Any<Address>()).Returns(true);

_stateProvider.GetBalance(_notAnAddress).Returns(0.Ether());
_stateProvider.AccountExists(_notAnAddress).Returns(false);
_stateProvider.IsContract(_notAnAddress).Returns(false);

_simulator.Simulate(Arg.Any<UserOperation>(), Arg.Any<BlockHeader>())
.ReturnsForAnyArgs(x => ResultWrapper<Keccak>.Success(Keccak.Zero));

_blockTree.Head.Returns(Core.Test.Builders.Build.A.Block.TestObject);

IPaymasterThrottler paymasterThrottler = Substitute.For<PaymasterThrottler>();

return new UserOperationPool(
config,
_blockTree,
entryPoint,
NullLogger.Instance,
paymasterThrottler,
_logFinder,
_signer,
_stateProvider,
Substitute.For<ITimestamper>(),
_simulator,
userOperationSortedPool,
1);
}

}
}
155 changes: 102 additions & 53 deletions ...tAbstraction.Test/AccountAbstractionRpcModuleTests.TestAccountAbstractionRpcBlockchain.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,14 @@
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using NUnit.Framework;
using FluentAssertions;
using Nethermind.Abi;
using Nethermind.AccountAbstraction.Contracts;
using Nethermind.AccountAbstraction.Data;
using Nethermind.AccountAbstraction.Executor;
using Nethermind.AccountAbstraction.Source;
using Nethermind.AccountAbstraction.Test.TestContracts;
using Nethermind.Blockchain;
using Nethermind.Blockchain.Comparers;
using Nethermind.Blockchain.Contracts.Json;
Expand Down Expand Up @@ -73,14 +75,13 @@ public static Task<TestAccountAbstractionRpcBlockchain> CreateChain(IReleaseSpec

public class TestAccountAbstractionRpcBlockchain : TestRpcBlockchain
{
public UserOperationPool UserOperationPool { get; private set; } = null!;
public UserOperationSimulator UserOperationSimulator { get; private set; } = null!;
public IDictionary<Address, UserOperationPool> UserOperationPool { get; private set; } = new Dictionary<Address, UserOperationPool>();
public IDictionary<Address, UserOperationSimulator> UserOperationSimulator { get; private set; } = new Dictionary<Address, UserOperationSimulator>();
public AbiDefinition EntryPointContractAbi { get; private set; } = null!;
public UserOperationTxBuilder UserOperationTxBuilder { get; private set; } = null!;
public IDictionary<Address, UserOperationTxBuilder> UserOperationTxBuilder { get; private set; } = new Dictionary<Address, UserOperationTxBuilder>();
public UserOperationTxSource UserOperationTxSource { get; private set; } = null!;

public Address EntryPointAddress { get; private set; } = null!;

public Address[] EntryPointAddresses { get; private set; } = null!;

public TestAccountAbstractionRpcBlockchain(UInt256? initialBaseFeePerGas)
{
Signer = new Signer(1, TestItem.PrivateKeyD, LogManager);
Expand All @@ -96,7 +97,7 @@ public TestAccountAbstractionRpcBlockchain(UInt256? initialBaseFeePerGas)
private AccountAbstractionConfig _accountAbstractionConfig = new AccountAbstractionConfig()
{
Enabled = true,
EntryPointContractAddress = "0xb0894727fe4ff102e1f1c8a16f38afc7b859f215",
EntryPointContractAddresses = "0xb0894727fe4ff102e1f1c8a16f38afc7b859f215,0x96cc609c8f5458fb8a7da4d94b678e38ebf3d04e",
Create2FactoryAddress = "0xd75a3a95360e44a3874e691fb48d77855f127069",
UserOperationPoolSize = 200
};
Expand All @@ -109,7 +110,7 @@ public TestAccountAbstractionRpcBlockchain(UInt256? initialBaseFeePerGas)
protected override IBlockProducer CreateTestBlockProducer(TxPoolTxSource txPoolTxSource, ISealer sealer, ITransactionComparerProvider transactionComparerProvider)
{
MiningConfig miningConfig = new() {MinGasPrice = UInt256.One};

BlockProducerEnvFactory blockProducerEnvFactory = new BlockProducerEnvFactory(
DbProvider,
BlockTree,
Expand All @@ -122,9 +123,17 @@ protected override IBlockProducer CreateTestBlockProducer(TxPoolTxSource txPoolT
TxPool,
transactionComparerProvider,
miningConfig,
LogManager);
LogManager)
{
TransactionsExecutorFactory =
new AABlockProducerTransactionsExecutorFactory(
SpecProvider,
LogManager,
Signer,
EntryPointAddresses)
};

UserOperationTxSource = new(UserOperationTxBuilder, UserOperationPool, UserOperationSimulator, SpecProvider, LogManager.GetClassLogger());
UserOperationTxSource = new(UserOperationTxBuilder, UserOperationPool, UserOperationSimulator, SpecProvider, State, Signer, LogManager.GetClassLogger());

Eth2TestBlockProducerFactory producerFactory = new Eth2TestBlockProducerFactory(GasLimitCalculator, UserOperationTxSource);
Eth2BlockProducer blockProducer = producerFactory.Create(
Expand All @@ -142,10 +151,18 @@ protected override IBlockProducer CreateTestBlockProducer(TxPoolTxSource txPoolT

protected override BlockProcessor CreateBlockProcessor()
{
Address.TryParse(_accountAbstractionConfig.EntryPointContractAddress, out Address? entryPointContractAddress);
// Address.TryParse(_accountAbstractionConfig.EntryPointContractAddress, out Address? entryPointContractAddress);
IList<Address> entryPointContractAddresses = new List<Address>();
IList<string> _entryPointContractAddressesString = _accountAbstractionConfig.GetEntryPointAddresses().ToList();
foreach (string _addressString in _entryPointContractAddressesString){
bool parsed = Address.TryParse(
_addressString,
out Address? entryPointContractAddress);
entryPointContractAddresses.Add(entryPointContractAddress!);
}

EntryPointAddresses = entryPointContractAddresses.ToArray();
Address.TryParse(_accountAbstractionConfig.Create2FactoryAddress, out Address? create2FactoryAddress);
EntryPointAddress = entryPointContractAddress!;

BlockValidator = CreateBlockValidator();
BlockProcessor blockProcessor = new(
SpecProvider,
Expand All @@ -163,52 +180,66 @@ protected override BlockProcessor CreateBlockProcessor()
var json = parser.LoadContract(typeof(EntryPoint));
EntryPointContractAbi = parser.Parse(json);

UserOperationTxBuilder = new UserOperationTxBuilder(
EntryPointContractAbi,
Signer,
entryPointContractAddress!,
SpecProvider,
State);
foreach(Address entryPoint in entryPointContractAddresses){
UserOperationTxBuilder[entryPoint] = new UserOperationTxBuilder(
EntryPointContractAbi,
Signer,
entryPoint!,
SpecProvider,
State);
}

UserOperationSimulator = new(
UserOperationTxBuilder,
State,
StateReader,
EntryPointContractAbi,
create2FactoryAddress!,
entryPointContractAddress!,
SpecProvider,
BlockTree,
DbProvider,
ReadOnlyTrieStore,
Timestamper,
LogManager);
foreach(Address entryPoint in entryPointContractAddresses){
UserOperationSimulator[entryPoint] = new(
UserOperationTxBuilder[entryPoint],
State,
StateReader,
EntryPointContractAbi,
create2FactoryAddress!,
entryPoint!,
SpecProvider,
BlockTree,
DbProvider,
ReadOnlyTrieStore,
Timestamper,
LogManager);
}

UserOperationPool = new UserOperationPool(
_accountAbstractionConfig,
BlockTree,
entryPointContractAddress!,
LogManager.GetClassLogger(),
new PaymasterThrottler(),
LogFinder,
Signer,
State,
Timestamper,
UserOperationSimulator,
new UserOperationSortedPool(
_accountAbstractionConfig.UserOperationPoolSize,
new CompareUserOperationsByDecreasingGasPrice(),
LogManager,
_accountAbstractionConfig.MaximumUserOperationPerSender),
SpecProvider.ChainId);
foreach(Address entryPoint in entryPointContractAddresses){
UserOperationPool[entryPoint] = new UserOperationPool(
_accountAbstractionConfig,
BlockTree,
entryPoint!,
LogManager.GetClassLogger(),
new PaymasterThrottler(),
LogFinder,
Signer,
State,
Timestamper,
UserOperationSimulator[entryPoint],
new UserOperationSortedPool(
_accountAbstractionConfig.UserOperationPoolSize,
new CompareUserOperationsByDecreasingGasPrice(),
LogManager,
_accountAbstractionConfig.MaximumUserOperationPerSender),
SpecProvider.ChainId);
}

return blockProcessor;
}

protected override async Task<TestBlockchain> Build(ISpecProvider? specProvider = null, UInt256? initialValues = null)
{
TestBlockchain chain = await base.Build(specProvider, initialValues);
AccountAbstractionRpcModule = new AccountAbstractionRpcModule(UserOperationPool, new []{new Address(_accountAbstractionConfig.EntryPointContractAddress)});
IList<Address> entryPointContractAddresses = new List<Address>();
IList<string> _entryPointContractAddressesString = _accountAbstractionConfig.GetEntryPointAddresses().ToList();
foreach (string _addressString in _entryPointContractAddressesString){
bool parsed = Address.TryParse(
_addressString,
out Address? entryPointContractAddress);
entryPointContractAddresses.Add(entryPointContractAddress!);
}
AccountAbstractionRpcModule = new AccountAbstractionRpcModule(UserOperationPool, entryPointContractAddresses.ToArray());

return chain;
}
Expand All @@ -227,11 +258,29 @@ private IBlockValidator CreateBlockValidator()

protected override Task AddBlocksOnStart() => Task.CompletedTask;

public void SendUserOperation(UserOperation userOperation)
public void SendUserOperation(Address entryPoint, UserOperation userOperation)
{
ResultWrapper<Keccak> resultOfUserOperation = UserOperationPool.AddUserOperation(userOperation);
ResultWrapper<Keccak> resultOfUserOperation = UserOperationPool[entryPoint].AddUserOperation(userOperation);
resultOfUserOperation.GetResult().ResultType.Should().NotBe(ResultType.Failure, resultOfUserOperation.Result.Error);
resultOfUserOperation.GetData().Should().Be(userOperation.CalculateRequestId(new Address(_accountAbstractionConfig.EntryPointContractAddress), SpecProvider.ChainId));
resultOfUserOperation.GetData().Should().Be(userOperation.CalculateRequestId(entryPoint, SpecProvider.ChainId));
}

public void SupportedEntryPoints()
{
ResultWrapper<Address[]> resultOfEntryPoints= AccountAbstractionRpcModule.eth_supportedEntryPoints();
resultOfEntryPoints.GetResult().ResultType.Should().NotBe(ResultType.Failure, resultOfEntryPoints.Result.Error);
IList<Address> entryPointContractAddresses = new List<Address>();
IList<string> _entryPointContractAddressesString = _accountAbstractionConfig.GetEntryPointAddresses().ToList();
foreach (string _addressString in _entryPointContractAddressesString){
bool parsed = Address.TryParse(
_addressString,
out Address? entryPointContractAddress);
entryPointContractAddresses.Add(entryPointContractAddress!);
}

Address[] eps = entryPointContractAddresses.ToArray();
Address[] recieved_eps = (Address[])(resultOfEntryPoints.GetData());
Assert.AreEqual(eps, recieved_eps);
}
}
}
Expand Down
Loading

0 comments on commit 114d0cc

Please sign in to comment.