Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acceptance DAS test + DAS fixes #179

Open
wants to merge 25 commits into
base: das
Choose a base branch
from

Conversation

Nashatyrev
Copy link
Owner

@Nashatyrev Nashatyrev commented Dec 30, 2024

PR Description

  • Add DAS acceptance test
  • Add Github workflow
  • c94a807 : When gossiped column is SAVE_FOR_FUTURE still broadcast it. Primarily for Custody to not download it again
    UPD: reverted and created separate issue: When gossiped column is SAVE_FOR_FUTURE, save it #180
  • 311e9a3: DasLongPollCustody: make gossip wait timeout depending on the slot duration to accommodate shorter slot configs (like 2 sec per slot)
  • 6c6e9c4: Add flush() to DataColumnSidecarRetriever and DataAvailabilitySampler to initiate the sampling without timeout implied by 1s delay in SimpleSidecarRetriever.netRound() loop

@Nashatyrev Nashatyrev changed the title das-pr/electra-acceptance-test Acceptance tests + DAS fixes Dec 30, 2024
@Nashatyrev Nashatyrev changed the title Acceptance tests + DAS fixes Acceptance DAS test + DAS fixes Dec 30, 2024
@Nashatyrev
Copy link
Owner Author

Relates to #164

Copy link
Collaborator

@zilm13 zilm13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test looks good and checks the data according to its purpose but I have some questions regarding implementation

@@ -65,7 +65,7 @@ public SafeFuture<InternalValidationResult> onDataColumnSidecarGossip(
return validation.thenPeek(
res -> {
dasGossipLogger.onReceive(dataColumnSidecar, res);
if (res.isAccept()) {
if (res.isAccept() || res.isSaveForFuture()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when it's safe for future there are a lot of checks missing, it's not valid

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right! Then it complicates things a lot 🤔
Need to think more about it then

Copy link
Owner Author

@Nashatyrev Nashatyrev Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted this change and filed a separate issue: #180

createTekuBeaconNode(
createConfigBuilder()
.withRealNetwork()
.withDasExtraCustodySubnetCount(128 - 4)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will cap it on 128 anyway
i'd replace this to 128 for readability

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it look better this way: 291cf5e ?

lateJoiningNode.start();
lateJoiningNode.waitForGenesis();
lateJoiningNode.waitUntilInSyncWith(primaryNode);
lateJoiningNode.waitForEpochAtOrAbove(6);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added this to check that sync is not lost after 2 epochs


int epochSlots = primaryNode.getSpec().slotsPerEpoch(UInt64.ZERO);
int endSlot = 6 * epochSlots;
assertAllBlocksExistWithoutForks(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this if they are in sync?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are checking that no blocks missed and no forks happened. Else it would mean that something went not that right

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nah I'm tolerating empty slots now. Probably worth to rename this method.
But we definitely don't expect forks here as we have just one validator node

@@ -30,6 +30,9 @@ public interface DataColumnSidecarRetriever {
*/
SafeFuture<DataColumnSidecar> retrieve(DataColumnSlotAndIdentifier columnId);

/** Starts all the queued retrieve requests immediately */
void flush();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure on this way to go. So we just call nextRound on this but it doesn't gurantee that the thing we submitted recently will be executed soon. It could be deep in the queue or whatever.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, probably the javadoc could be more accurate? Or do you have any other approach in mind?

@Nashatyrev Nashatyrev force-pushed the das-pr/electra-acceptance-test branch from 291cf5e to 607aebf Compare January 22, 2025 09:34
@Nashatyrev Nashatyrev changed the base branch from das-pr/electra to das January 22, 2025 09:35
@Nashatyrev Nashatyrev marked this pull request as ready for review January 22, 2025 10:37
…rimarily for Custody to not download it again"

This reverts commit 916c126.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants