Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trace/db traces #4535

Merged
merged 81 commits into from
Dec 6, 2022
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
3ccad6b
Implement TraceStore plugin
LukaszRozmej Sep 5, 2022
7a6cf09
Add compression to db
LukaszRozmej Sep 6, 2022
1bdb448
Allow unsafe to avoid allocation
LukaszRozmej Sep 6, 2022
ad7af10
Fix EndBlockTrace to be after Applying rewards
LukaszRozmej Sep 7, 2022
f17a16a
Add ByteArrayPoolList
LukaszRozmej Sep 7, 2022
3bd79f0
Make ResettableList implement IReadOnlyCollection
LukaszRozmej Sep 7, 2022
e02e1b1
Make DbOnTheRocks default to nameless configuration if named not found
LukaszRozmej Sep 7, 2022
44ddf92
Make BlockTracerBase based on ResettableList
LukaszRozmej Sep 7, 2022
3a21382
Fixes and improvements
LukaszRozmej Sep 7, 2022
baf65d5
Add plugin to pipelines
LukaszRozmej Sep 7, 2022
13a0f38
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Sep 7, 2022
03eb50d
fix test
LukaszRozmej Sep 7, 2022
9190ed0
Add temporary traces
LukaszRozmej Sep 7, 2022
11d946c
Revert "Add temporary traces"
LukaszRozmej Sep 7, 2022
d51646a
Add exception data to json rpc errors
LukaszRozmej Sep 7, 2022
425faf2
Increase MaxDepth to 128
LukaszRozmej Sep 7, 2022
7e86dab
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Sep 16, 2022
c40030b
Refactor product versioning and revise Docker files (#4781)
rubo Oct 20, 2022
cef1f72
Upgrade version to 14.4 (#4796)
kamilchodola Oct 21, 2022
483220c
Chiado TTD (#4801)
jmederosalvarado Oct 24, 2022
0be02d8
Updating Fast Sync config files (#4803)
github-actions[bot] Oct 24, 2022
161d78e
Feature: added extra data length checks (#4805)
OlegJakushkin Oct 24, 2022
582cb9e
Fix workflow to pass parameters to scripts (#4799)
rubo Oct 24, 2022
c174ad5
Autorecover when disconnected chain is found (#4816)
asdacap Oct 26, 2022
d7f8306
Updating Fast Sync config files (#4819)
github-actions[bot] Oct 26, 2022
e77b722
Fix environment variable definition (#4825)
rubo Oct 26, 2022
280c9ab
Revert "Fix sepolia invalid block (#4694)" (#4838)
MarekM25 Oct 28, 2022
380bf9c
Bump build version to 14.5 (#4839)
kamilchodola Oct 28, 2022
e4780ca
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Oct 28, 2022
d48d4ce
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Oct 31, 2022
e0082af
fix transition creation
LukaszRozmej Oct 31, 2022
ed10b34
Merge remote-tracking branch 'origin/fix/timestamp_activation' into t…
LukaszRozmej Oct 31, 2022
da17fc1
Don't log error, when initialization is stopped (like when syncing pr…
LukaszRozmej Nov 10, 2022
31ff457
Add VerifySerialized option
LukaszRozmej Nov 10, 2022
96b8756
Add serialization tests
LukaszRozmej Nov 11, 2022
65870a4
Move VerifySerialized to TraceSerializer, add MaxDepth config
LukaszRozmej Nov 11, 2022
8f2fe84
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Nov 11, 2022
501a4d9
fix whitespace
LukaszRozmej Nov 11, 2022
c0e802e
fix null handling in TxBroadcaster
LukaszRozmej Nov 11, 2022
a3c84e7
Refactor ITraceSerializer
LukaszRozmej Nov 14, 2022
e9e55c1
fix test
LukaszRozmej Nov 14, 2022
5b7ef97
undo fix
LukaszRozmej Nov 14, 2022
463810b
whitespace fix
LukaszRozmej Nov 14, 2022
921b2ff
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Nov 14, 2022
2a113b9
test fix
LukaszRozmej Nov 14, 2022
7a4a37b
fixes
LukaszRozmej Nov 14, 2022
c1e61c5
Set chiado and xdai `Merge.SecondsPerSlot=5` (#4880)
jmederosalvarado Nov 9, 2022
17b3048
Updating Fast Sync config files (#4897)
github-actions[bot] Nov 13, 2022
045200e
Set C# version to 10.0 (#4881)
rubo Nov 9, 2022
6f8b889
Add gnosischain TTD value (#4901)
jmederosalvarado Nov 15, 2022
dbf415b
fix eth_call mix_hash and beneficiary (#4873)
LukaszRozmej Nov 15, 2022
acd3a42
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Nov 17, 2022
c9047bf
Changes the docker password to a token (#4916)
FalcoXYZ Nov 17, 2022
fdc1a10
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Nov 17, 2022
2451401
parallelized trace_filter execution per block
LukaszRozmej Nov 17, 2022
d75401d
Remove unneded filters
LukaszRozmej Nov 17, 2022
4ef38f5
Reduce traces memory allocation
LukaszRozmej Nov 17, 2022
30c9d70
fix trace filtering
LukaszRozmej Nov 17, 2022
0adba53
fix whitespace
LukaszRozmej Nov 18, 2022
2e74b93
Updating Fast Sync config files (#4920)
github-actions[bot] Nov 21, 2022
8a30004
Bump build version to 14.6 (#4923)
kamilchodola Nov 21, 2022
6aea772
Changes the docker password variable to token (#4924)
FalcoXYZ Nov 21, 2022
84758ab
null action result when it is empty
LukaszRozmej Nov 28, 2022
b42a006
Merge remote-tracking branch 'origin/release/1.14.6' into trace/db-tr…
LukaszRozmej Nov 28, 2022
b6261c7
Temporarily switch to `ubuntu-20.04` image (#4927)
LukaszRozmej Nov 28, 2022
a458eb9
Ignore ParityResult.IsEmpty in JSON
LukaszRozmej Nov 28, 2022
08207dc
Fix remove empty results
LukaszRozmej Nov 29, 2022
a9a05fe
Merge 1.14.7
LukaszRozmej Dec 1, 2022
778780f
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Dec 5, 2022
87f1b50
fix licensing header
LukaszRozmej Dec 5, 2022
3952d3e
fix whitespace
LukaszRozmej Dec 5, 2022
73b1071
Final touches
LukaszRozmej Dec 5, 2022
2305f2a
Make ITraceSerializer generic
LukaszRozmej Dec 6, 2022
44eb3e0
refactor
LukaszRozmej Dec 6, 2022
ba193cf
Merge remote-tracking branch 'origin/master' into trace/db-traces
LukaszRozmej Dec 6, 2022
7bff0b8
Adds IsTracingFees docs
LukaszRozmej Dec 6, 2022
601bd16
FIx issue when Result was added to json, because it was created by de…
LukaszRozmej Dec 6, 2022
d100122
fix shutdown if TraceStorePlugin not Enabled
LukaszRozmej Dec 6, 2022
210a501
Review fixes
LukaszRozmej Dec 6, 2022
686f15d
Fix tests
LukaszRozmej Dec 6, 2022
7a5647b
fix shitespace
LukaszRozmej Dec 6, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,12 @@ jobs:
- name: Nethermind.Sockets.Test
run: |
dotnet test -c Release $EXCLUDE_TEST_PROJECTS src/Nethermind/Nethermind.Sockets.Test
- name: Nethermind.JsonRpc.TraceStore.Tests
run: |
dotnet test -c Release $EXCLUDE_TEST_PROJECTS src/Nethermind/Nethermind.JsonRpc.TraceStore.Tests
- name: MathGmp.Native
run: |
dotnet test -c Release $EXCLUDE_TEST_PROJECTS src/Math.Gmp.Native/MathGmp.Native.UnitTests
dotnet test -c Release $EXCLUDE_TEST_PROJECTS src/Math.Gmp.Native/MathGmp.Native.UnitTests
- name: Upload Codecov Report
if: matrix.os == 'ubuntu-latest'
uses: actions/upload-artifact@v2
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/run-nethermind-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ jobs:
- name: Nethermind.AccountAbstraction.Test
run: |
dotnet test -c Release src/Nethermind/Nethermind.AccountAbstraction.Test
- name: Nethermind.JsonRpc.TraceStore.Tests
run: |
dotnet test -c Release src/Nethermind/Nethermind.JsonRpc.TraceStore.Tests

neth-tests2:
name: Running Nethermind Tests 2
Expand Down
10 changes: 5 additions & 5 deletions scripts/deployment/archive-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ dotnet build -c Release Nethermind.sln

cd $RELEASE_DIRECTORY

cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev}.dll $LIN_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev}.dll $OSX_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev}.dll $WIN_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev}.dll $LIN_ARM64_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev}.dll $OSX_ARM64_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev,JsonRpc.TraceStore}.dll $LIN_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev,JsonRpc.TraceStore}.dll $OSX_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev,JsonRpc.TraceStore}.dll $WIN_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev,JsonRpc.TraceStore}.dll $LIN_ARM64_RELEASE/plugins
cp $RELEASE_DIRECTORY/$RELEASE_PATH/Nethermind.{Api,HealthChecks,EthStats,Merge.Plugin,Mev,JsonRpc.TraceStore}.dll $OSX_ARM64_RELEASE/plugins

cd $LIN_RELEASE && zip -r $LIN-$GIT_SHORT_TAG-$GIT_HASH.zip . && cd ..
cd $OSX_RELEASE && zip -r $OSX-$GIT_SHORT_TAG-$GIT_HASH.zip . && cd ..
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,10 @@ protected virtual TxReceipt[] ProcessBlock(
_receiptsTracer.SetOtherTracer(blockTracer);
_receiptsTracer.StartNewBlockTrace(block);
TxReceipt[] receipts = _blockTransactionsExecutor.ProcessTransactions(block, options, _receiptsTracer, spec);
_receiptsTracer.EndBlockTrace();

block.Header.ReceiptsRoot = receipts.GetReceiptsRoot(spec, block.ReceiptsRoot);
ApplyMinerRewards(block, blockTracer, spec);
_receiptsTracer.EndBlockTrace();

_stateProvider.Commit(spec);
_stateProvider.RecalculateStateRoot();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
// Copyright (c) 2021 Demerzel Solutions Limited
// This file is part of the Nethermind library.
//
//
// The Nethermind library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
//
// The Nethermind library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
//
// You should have received a copy of the GNU Lesser General Public License
// along with the Nethermind. If not, see <http://www.gnu.org/licenses/>.
//
//

using System;
using System.Linq;
Expand All @@ -35,15 +35,15 @@ public void Empty_list()
list.Capacity.Should().Be(1024);
list.IsReadOnly.Should().BeFalse();
}

[Test]
public void Add_should_work()
{
ArrayPoolList<int> list = new(1024);
list.AddRange(Enumerable.Range(0, 4));
list.Should().BeEquivalentTo(Enumerable.Range(0, 4));
}

[Test]
public void Add_should_expand()
{
Expand All @@ -53,7 +53,7 @@ public void Add_should_expand()
list.Count.Should().Be(50);
list.Capacity.Should().Be(64);
}

[Test]
public void Clear_should_clear()
{
Expand All @@ -64,7 +64,7 @@ public void Clear_should_clear()
list.Count.Should().Be(0);
list.Capacity.Should().Be(64);
}

[TestCase(0, ExpectedResult = true)]
[TestCase(20, ExpectedResult = true)]
[TestCase(100, ExpectedResult = false)]
Expand All @@ -75,7 +75,7 @@ public bool Contains_should_check_ok(int item)
list.AddRange(Enumerable.Range(0, 50));
return list.Contains(item);
}

[TestCase(0, new[] {-1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15})]
[TestCase(4, new[] {0, 1, 2, 3, -1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15})]
[TestCase(16, new[] {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, -1})]
Expand All @@ -96,7 +96,7 @@ public void Insert_should_throw(int index)
Action action = () => list.Insert(index, -1);
action.Should().Throw<ArgumentOutOfRangeException>();
}

[TestCase(0, ExpectedResult = 0)]
[TestCase(40, ExpectedResult = 40)]
[TestCase(50, ExpectedResult = -1)]
Expand All @@ -107,8 +107,8 @@ public int IndexOf_should_return_index(int item)
list.AddRange(Enumerable.Range(0, 50));
return list.IndexOf(item);
}


[TestCase(0, true, new[] {1, 2, 3, 4, 5, 6, 7})]
[TestCase(7, true, new[] {0, 1, 2, 3, 4, 5, 6})]
[TestCase(8, false, new[] {0, 1, 2, 3, 4, 5, 6, 7})]
Expand All @@ -120,7 +120,7 @@ public void Remove_should_remove(int item, bool removed, int[] expected)
list.Remove(item).Should().Be(removed);
list.Should().BeEquivalentTo(expected);
}

[TestCase(0, new[] {1, 2, 3, 4, 5, 6, 7})]
[TestCase(7, new[] {0, 1, 2, 3, 4, 5, 6})]
public void RemoveAt_should_remove(int item, int[] expected)
Expand All @@ -130,7 +130,7 @@ public void RemoveAt_should_remove(int item, int[] expected)
list.RemoveAt(item);
list.Should().BeEquivalentTo(expected);
}

[TestCase(8, new[] {0, 1, 2, 3, 4, 5, 6, 7})]
[TestCase(-1, new[] {0, 1, 2, 3, 4, 5, 6, 7})]
public void RemoveAt_should_throw(int item, int[] expected)
Expand All @@ -140,7 +140,7 @@ public void RemoveAt_should_throw(int item, int[] expected)
Action action = () => list.RemoveAt(item);
action.Should().Throw<ArgumentOutOfRangeException>();
}

[Test]
public void CopyTo_should_copy()
{
Expand All @@ -150,7 +150,7 @@ public void CopyTo_should_copy()
list.CopyTo(array, 1);
array.Should().BeEquivalentTo(Enumerable.Range(0, 1).Concat(Enumerable.Range(0, 50)));
}

[TestCase(0, ExpectedResult = 0)]
[TestCase(7, ExpectedResult = 7)]
public int Get_should_return(int item)
Expand All @@ -159,7 +159,7 @@ public int Get_should_return(int item)
list.AddRange(Enumerable.Range(0, 8));
return list[item];
}

[TestCase(8)]
[TestCase(-1)]
public void Get_should_throw(int item)
Expand All @@ -169,7 +169,7 @@ public void Get_should_throw(int item)
Func<int> action = () => list[item];
action.Should().Throw<ArgumentOutOfRangeException>();
}

[TestCase(0, ExpectedResult = -1)]
[TestCase(7, ExpectedResult = -1)]
public int Set_should_set(int item)
Expand All @@ -179,7 +179,7 @@ public int Set_should_set(int item)
list[item] = -1;
return list[item];
}

[TestCase(8)]
[TestCase(-1)]
public void Set_should_throw(int item)
Expand All @@ -189,5 +189,18 @@ public void Set_should_throw(int item)
Action action = () => list[item] = 1;
action.Should().Throw<ArgumentOutOfRangeException>();
}

[TestCase(1, 16)]
[TestCase(14, 16)]
[TestCase(15, 32)]
[TestCase(20, 32)]
[TestCase(100, 128)]
public void AddRange_should_expand(int items, int expectedCapacity)
{
ArrayPoolList<int> list = new(16) { 0, 1 };
list.AddRange(Enumerable.Range(2, items));
list.Should().BeEquivalentTo(Enumerable.Range(0, items + 2));
list.Capacity.Should().Be(expectedCapacity);
}
}
}
22 changes: 12 additions & 10 deletions src/Nethermind/Nethermind.Core/BlockInfo.cs
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
// Copyright (c) 2021 Demerzel Solutions Limited
// This file is part of the Nethermind library.
//
//
// The Nethermind library is free software: you can redistribute it and/or modify
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
//
// The Nethermind library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Lesser General Public License for more details.
//
//
// You should have received a copy of the GNU Lesser General Public License
// along with the Nethermind. If not, see <http://www.gnu.org/licenses/>.

Expand All @@ -30,7 +30,7 @@ public enum BlockMetadata
BeaconBody = 8,
BeaconMainChain = 16
}

public class BlockInfo
{
public BlockInfo(Keccak blockHash, in UInt256 totalDifficulty, BlockMetadata metadata = BlockMetadata.None)
Expand All @@ -39,9 +39,9 @@ public BlockInfo(Keccak blockHash, in UInt256 totalDifficulty, BlockMetadata met
TotalDifficulty = totalDifficulty;
Metadata = metadata;
}

public UInt256 TotalDifficulty { get; set; }

public bool WasProcessed { get; set; }

public Keccak BlockHash { get; }
Expand All @@ -61,8 +61,8 @@ public bool IsFinalized
}
}
}


public bool IsBeaconHeader
{
get => (Metadata & BlockMetadata.BeaconHeader) != 0;
Expand All @@ -72,12 +72,12 @@ public bool IsBeaconBody
{
get => (Metadata & BlockMetadata.BeaconBody) != 0;
}

public bool IsBeaconMainChain
{
get => (Metadata & BlockMetadata.BeaconMainChain) != 0;
}

public bool IsBeaconInfo
{
get => (Metadata & (BlockMetadata.BeaconBody | BlockMetadata.BeaconHeader)) != 0;
Expand All @@ -89,5 +89,7 @@ public bool IsBeaconInfo
/// This property is not serialized
/// </summary>
public long BlockNumber { get; set; }

public override string ToString() => BlockHash.ToString();
}
}
Loading