Full archive peer shard mapper #5337

sstanculeanu · 2023-06-12T15:59:03Z

Reasoning behind the pull request

new peer shard mapper needed for the new messenger

Proposed changes

created and integrated a new peer shard mapper

Testing procedure

will be tested with the full feat branch

Pre-requisites

Based on the Contributing Guidelines the PR author and the reviewers must check the following requirements are met:

was the PR targeted to the correct branch?
if this is a larger feature that probably needs more than one PR, is there a feat branch created?
if this is a feat branch merging, do all satellite projects have a proper tag inside go.mod?

small refactor on antiflood components creation created a new full history peer shard mapper create new interceptors for heartbeat to feed the new peer shard mapper

…or_fullarchive_messenger

ssd04 · 2023-06-14T14:15:47Z

node/nodeHelper.go

+	peerDenialEvaluator, err := preparePeerDenialEvaluators(networkComponents, processComponents)
 	if err != nil {


so we don't need to return full achive peer denial evaluator and set it when creating NewNode?

nope.. the peer denial evaluator at node level is used only to check if a peer is denied, which relays on 2 shared components for both regular denial evaluator and full archive one.. so the response would be the same

ssd04 · 2023-06-14T14:19:39Z

process/factory/interceptorscontainer/baseInterceptorsContainerFactory.go

+	return bicf.container.Add(identifierHeartbeat, interceptor)
+}
+
+func (bicf *baseInterceptorsContainerFactory) createOneHeartbeatV2Interceptor(


Suggested change

func (bicf *baseInterceptorsContainerFactory) createOneHeartbeatV2Interceptor(

func (bicf *baseInterceptorsContainerFactory) createHeartbeatV2Interceptor(

?

ssd04 · 2023-06-14T14:20:04Z

process/factory/interceptorscontainer/baseInterceptorsContainerFactory.go

+	return bicf.container.Add(identifier, interceptor)
+}
+
+func (bicf *baseInterceptorsContainerFactory) createOnePeerShardInterceptor(


Suggested change

func (bicf *baseInterceptorsContainerFactory) createOnePeerShardInterceptor(

func (bicf *baseInterceptorsContainerFactory) createPeerShardInterceptor(

?

common/disabled/cache.go

epochStart/bootstrap/fromLocalStorage.go

epochStart/bootstrap/process.go

epochStart/bootstrap/storageProcess.go

factory/processing/processComponents.go

iulianpascalau · 2023-06-15T06:07:55Z

integrationTests/state/stateTrieSync/stateTrieSync_test.go

@@ -53,7 +53,8 @@ func createTestProcessorNodeAndTrieStorage(
 		TrieStore:            mainStorer,
 		GasScheduleMap:       createTestGasMap(),
 	})
-	_ = node.Messenger.CreateTopic(common.ConsensusTopic+node.ShardCoordinator.CommunicationIdentifier(node.ShardCoordinator.SelfId()), true)
+	_ = node.MainMessenger.CreateTopic(common.ConsensusTopic+node.ShardCoordinator.CommunicationIdentifier(node.ShardCoordinator.SelfId()), true)
+	_ = node.FullArchiveMessenger.CreateTopic(common.ConsensusTopic+node.ShardCoordinator.CommunicationIdentifier(node.ShardCoordinator.SelfId()), true)


not required

If this doesn't panic, do we need to close all messengers on the defer functions? (valid on all occurrences)
Maybe create a function on the test processor node that will close all messengers?

the test processor node already has a Close() method that closes all messengers.. updated tests to call it

common/constants.go

node/nodeHelper.go

process/factory/interceptorscontainer/baseInterceptorsContainerFactory.go

ssd04 · 2023-06-15T14:54:16Z

integrationTests/state/stateTrieSync/stateTrieSync_test.go

@@ -254,7 +254,7 @@ func testNodeRequestInterceptTrieNodesWithMessengerNotSyncingShouldErr(t *testin
 	go func() {
 		// sudden close of the resolver node after just 2 seconds
 		time.Sleep(time.Second * 2)
-		_ = nResolver.Messenger.Close()
+		_ = nResolver.MainMessenger.Close()


Suggested change

_ = nResolver.MainMessenger.Close()

nResolver.Close()

?

update all tests

codecov · 2023-06-15T15:17:24Z

Codecov Report

Patch coverage: 83.41% and project coverage change: +0.07 🎉

Comparison is base (c28cf02) 80.26% compared to head (cbeb97d) 80.34%.

❗ Current head cbeb97d differs from pull request most recent head 11cdfc9. Consider uploading reports for the commit 11cdfc9 to get more accurate results

Additional details and impacted files

@@                       Coverage Diff                        @@
##           feat/multiple_p2p_messengers    #5337      +/-   ##
================================================================
+ Coverage                         80.26%   80.34%   +0.07%     
================================================================
  Files                               695      695              
  Lines                             90244    90370     +126     
================================================================
+ Hits                              72432    72604     +172     
+ Misses                            12631    12599      -32     
+ Partials                           5181     5167      -14

Impacted Files	Coverage Δ
epochStart/bootstrap/fromLocalStorage.go	`49.15% <0.00%> (-1.14%)`	⬇️
factory/network/networkComponentsHandler.go	`71.65% <0.00%> (ø)`
...tiflood/factory/p2pAntifloodAndBlacklistFactory.go	`73.98% <0.00%> (-3.12%)`	⬇️
node/nodeHelper.go	`71.11% <59.37%> (-6.84%)`	⬇️
factory/processing/processComponentsHandler.go	`81.00% <63.63%> (-0.30%)`	⬇️
epochStart/bootstrap/process.go	`84.31% <74.28%> (-0.52%)`	⬇️
factory/network/networkComponents.go	`87.95% <78.12%> (+1.98%)`	⬆️
...ptorscontainer/baseInterceptorsContainerFactory.go	`77.51% <87.83%> (+2.77%)`	⬆️
factory/processing/processComponents.go	`82.38% <94.91%> (-0.12%)`	⬇️
epochStart/bootstrap/common.go	`100.00% <100.00%> (ø)`
... and 5 more

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

iulianpascalau · 2023-06-15T13:31:25Z

epochStart/bootstrap/factory/epochStartInterceptorsContainerFactory.go

 	}

-	return container, nil
+	if args.NodeOperationMode == p2p.FullArchiveMode {
+		err = interceptorsContainerFactory.AddShardTrieNodeInterceptors(fullArchiveContainer)


can be left as it is but we do not support trie sync in an arbitrary epoch. Debatable if we will support at a certain time.

further integration for the full archive messenger

b06188a

small refactor on antiflood components creation created a new full history peer shard mapper create new interceptors for heartbeat to feed the new peer shard mapper

sstanculeanu added area:processing impact:medium interested:protocol type:improvement complexity:medium labels Jun 12, 2023

sstanculeanu self-assigned this Jun 12, 2023

Base automatically changed from full_archive_heartbeat_sender to feat/multiple_p2p_messengers June 14, 2023 09:54

Merge branch 'feat/multiple_p2p_messengers' into process_components_f…

74b5b02

…or_fullarchive_messenger

sstanculeanu marked this pull request as ready for review June 14, 2023 09:56

iulianpascalau self-requested a review June 14, 2023 10:04

ssd04 self-requested a review June 14, 2023 13:48

ssd04 reviewed Jun 14, 2023

View reviewed changes

iulianpascalau reviewed Jun 15, 2023

View reviewed changes

fixes after review, added new interceptors container

633fdf7

ssd04 reviewed Jun 15, 2023

View reviewed changes

replaced messenger.Close with node.Close on tests

11cdfc9

iulianpascalau approved these changes Jun 16, 2023

View reviewed changes

ssd04 approved these changes Jun 16, 2023

View reviewed changes

sstanculeanu merged commit 1e22b2d into feat/multiple_p2p_messengers Jun 16, 2023

sstanculeanu deleted the process_components_for_fullarchive_messenger branch June 16, 2023 09:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full archive peer shard mapper #5337

Full archive peer shard mapper #5337

sstanculeanu commented Jun 12, 2023

ssd04 Jun 14, 2023

sstanculeanu Jun 15, 2023

ssd04 Jun 14, 2023

ssd04 Jun 14, 2023

iulianpascalau Jun 15, 2023

sstanculeanu Jun 15, 2023

ssd04 Jun 15, 2023

sstanculeanu Jun 15, 2023

codecov bot commented Jun 15, 2023

iulianpascalau Jun 15, 2023

		peerDenialEvaluator, err := preparePeerDenialEvaluators(networkComponents, processComponents)
		if err != nil {

	func (bicf *baseInterceptorsContainerFactory) createOneHeartbeatV2Interceptor(
	func (bicf *baseInterceptorsContainerFactory) createHeartbeatV2Interceptor(

Full archive peer shard mapper #5337

Full archive peer shard mapper #5337

Conversation

sstanculeanu commented Jun 12, 2023

Reasoning behind the pull request

Proposed changes

Testing procedure

Pre-requisites

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 15, 2023

Codecov Report

Choose a reason for hiding this comment