Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to network discv5 boot enrs on start up #3672

Merged
merged 9 commits into from
Feb 4, 2022
Merged

Connect to network discv5 boot enrs on start up #3672

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
0e26ca5
temp
dadepo Jan 25, 2022
d100709
temp
dadepo Jan 26, 2022
a8efda7
Ensure the values passed via network.discv5.bootEnrs are connected to…
dadepo Jan 27, 2022
6e79bc5
updated the documentation to include setting enr.ip for first node
dadepo Feb 2, 2022
dcacf7f
Removed cli flags not needed when using dev command in the example
dadepo Feb 4, 2022
38c0b02
put connecting to disc5v behind a cli flag
dadepo Feb 4, 2022
034b8ba
Removed a TODO comment to be processed in another PR
dadepo Feb 4, 2022
ad28754
Updated documentation to include the needed network.connectToDiscv5Bo…
dadepo Feb 4, 2022
9c5362f
Update packages/lodestar/src/network/peers/discover.ts
wemeetagain Feb 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 53 additions & 35 deletions docs/usage/local.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,58 +1,76 @@
# Local testnet

To quickly test and run Lodestar we recommend to start a local testnet. We recommend a simple configuration of two beacon nodes with multiple validators
To quickly test and run Lodestar we recommend starting a local testnet. We recommend a simple configuration of two beacon nodes with multiple validators

**Terminal 1**

Run a beacon node with 8 validators and default settings. State will be written to .tmp/state.ssz
Run a beacon node, with 8 validators with the following command.

```bash
./lodestar dev --genesisValidators 8 --reset
./lodestar dev --genesisValidators 8 --genesisTime 1578787200 --network prater --rootDir </path/to/node1> --reset
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe remove --network prater here and below

I think this is a little confusing as an example in the docs, folks may think they are actually connecting to the prater testnet

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be network option can be removed from the dev cli options, so that it will throw error if someone even tries to specify

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be network option can be removed from the dev cli options, so that it will throw error if someone even tries to specify

Good tip. I'll see to that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech so I looked into your suggestion. The network option is a global option and hence the changes that will be required to make it not applicable when the dev cli option is used feels a bit out of scope to this PR. So I created a separate issue: #3695 for the suggestion to be implemented separately.

```

`--genesisValidators` and `--genesisTime` define the genesis state of the beacon chain. `--rootDir` defines a path where lodestar should store the beacon state
while the `--reset` flag ensures the state is cleared on each restart - which is useful when testing locally.

Once the node has started, make a request to `curl http://localhost:9596/eth/v1/node/identity` and copy the `enr` value.

This would be used to connect from the second node.

> enr stands for ethereum node records, which is a format for conveying p2p connectivity information for ethereum nodes.
> For more info see [eip-778](https://eips.ethereum.org/EIPS/eip-778)

**Terminal 2**

Connect to bootnode (node 1 default multiaddrs) but without starting validators.
Start the second node without starting any validators and connect to the first node by supplying the copied `enr` value:

```bash
./lodestar dev --startValidators 0:0 \
--genesisStateFile ./dev/genesis.ssz \
--network.localMultiaddrs /ip4/127.0.0.1/tcp/30607 \
--sync.minPeers 1
--genesisValidators 8 --genesisTime 1578787200 \
--network prater \
--rootDir /path/to/node2 \
--port 9001 \
--api.rest.port 9597 \
--network.discv5.bootEnrs <enr value>
--reset
Copy link
Contributor

@g11tech g11tech Feb 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be this will now need connectToDiscv5Bootnodes arg as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Updated.

```

---
By default, lodestar starts as many validators as the number supplied by `--genesisValidators`. In other to not start any validator, this is overridden by
the `--startValidators` option. Passing a value of `0:0` means no validators should be started.

Once both instances are running you should see an output similar to this
Also, take note that the values of `--genesisValidators` and `--genesisTime` must be the same as the ones passed to the first node in other for the two nodes
to have the same beacon chain.

**Terminal 1**
Finally `port` and `api.rest.port` are supplied since the default values will already be in use by the first node.

Once the second node starts, you should see an output similar to the following in either of the terminals:

```bash
2020-06-21 16:42:40 [SYNC] warn: Current peerCount=0, required = 2
2020-06-21 16:42:43 [SYNC] warn: Current peerCount=0, required = 2
2020-06-21 16:42:43 [VALIDATOR 7] info: Validator is proposer at slot 9
2020-06-21 16:42:43 [CHAIN] info: Processed new chain head newChainHeadRoot=0x3223a51b51fa4f42ea2281e8580806907a8e69f490cfe12a380b8e8b41b21d27, slot=9, epoch=1
2020-06-21 16:42:43 [VALIDATOR 7] info: Proposed block with hash 0x3223a51b51fa4f42ea2281e8580806907a8e69f490cfe12a380b8e8b41b21d27 and slot 9
2020-06-21 16:42:46 [SYNC] warn: Current peerCount=1, required = 2
2020-06-21 16:42:49 [SYNC] warn: Current peerCount=1, required = 2
2020-06-21 16:42:49 [VALIDATOR 6] info: Validator is proposer at slot 10
2020-06-21 16:42:49 [CHAIN] info: Processed new chain head newChainHeadRoot=0x7c3a77ad892ca631b750b988277a6caca9cc011461e326537fb607c94359b95f, slot=10, epoch=1
2020-06-21 16:42:49 [VALIDATOR 6] info: Proposed block with hash 0x7c3a77ad892ca631b750b988277a6caca9cc011461e326537fb607c94359b95f and slot 10
2020-06-21 16:42:52 [SYNC] warn: Current peerCount=1, required = 2
2020-06-21 16:42:55 [SYNC] warn: Current peerCount=1, required = 2
2020-06-21 16:42:55 [VALIDATOR 2] info: Validator is proposer at slot 11
2020-06-21 16:42:55 [CHAIN] info: Processed new chain head newChainHeadRoot=0x74e96be4058e0edec26028c2f727b30dbc05e12c3f29f364e487916e16777f4a, slot=11, epoch=1
2020-06-21 16:42:55 [VALIDATOR 2] info: Proposed block with hash 0x74e96be4058e0edec26028c2f727b30dbc05e12c3f29f364e487916e16777f4a and slot 11
```
Eph 167991/6 6.007 [] info: Searching peers - peers: 1 - slot: 5375718 (skipped 5375718) - head: 0 0xcc67…3345 - finalized: 0x0000…0000:0
```

**Terminal 2**
For further confirmation that both nodes are connected as peers, make a request to the `/eth/v1/node/peers` endpoint.

For example, making the request on the first node via the following command:

`curl http://localhost:9596/eth/v1/node/peers | jq`

will give a result similar to the following:

```bash
2020-06-21 16:42:49 [SYNC] info: Sync caught up to latest slot 9
2020-06-21 16:42:49 [CHAIN] info: Processed new chain head newChainHeadRoot=0x7c3a77ad892ca631b750b988277a6caca9cc011461e326537fb607c94359b95f, slot=10, epoch=1
2020-06-21 16:42:56 [CHAIN] info: Processed new chain head newChainHeadRoot=0x74e96be4058e0edec26028c2f727b30dbc05e12c3f29f364e487916e16777f4a, slot=11, epoch=1
2020-06-21 16:43:01 [CHAIN] info: Processed new chain head newChainHeadRoot=0x75ff1e7143acead878913a516c87f620022e178298e7a7f4a9485fd731bc7128, slot=12, epoch=1
2020-06-21 16:43:07 [CHAIN] info: Processed new chain head newChainHeadRoot=0x250da6f5ebad021894eb07824b535c3442fe0f7a67949f266d46ffa6b5a18b76, slot=13, epoch=1
2020-06-21 16:43:13 [CHAIN] info: Processed new chain head newChainHeadRoot=0x2e90c4a2cea722cb8bfccdfe3b73a4211e7a21d07075d11307626d8b048b9074, slot=14, epoch=1
2020-06-21 16:43:19 [CHAIN] info: Processed new chain head newChainHeadRoot=0x03fcff4f23de519c1e294f6b1256d194199c107c56b4466efed6bfab8d6e7e92, slot=15, epoch=1
```

{
"data": [
{
"peer_id": "...",
"enr": "",
"last_seen_p2p_address": "....",
"direction": "inbound",
"state": "connected"
}
],
"meta": {
"count": 1
}
}
```
2 changes: 1 addition & 1 deletion packages/cli/src/cmds/dev/handler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ export async function devHandler(args: IDevArgs & IGlobalArgs): Promise<void> {
if (args.logFormatGenesisTime === undefined) args.logFormatGenesisTime = genesisTime;

// BeaconNode setup
const libp2p = await createNodeJsLibp2p(peerId, options.network);
const libp2p = await createNodeJsLibp2p(peerId, options.network, {peerStoreDir: beaconPaths.peerStoreDir});
const logger = getCliLogger(args, beaconPaths, config);
logger.info("Lodestar", {version: getVersion(), network: args.network});
if (ACTIVE_PRESET === PresetName.minimal) logger.info("ACTIVE_PRESET == minimal preset");
Expand Down
9 changes: 6 additions & 3 deletions packages/lodestar/src/network/nodejs/util.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,12 @@ export async function createNodeJsLibp2p(

// Append discv5.bootEnrs to bootMultiaddrs if requested
if (networkOpts.connectToDiscv5Bootnodes) {
if (!networkOpts.bootMultiaddrs) networkOpts.bootMultiaddrs = [];
if (!networkOpts.discv5) networkOpts.discv5 = defaultDiscv5Options;

if (!networkOpts.bootMultiaddrs) {
networkOpts.bootMultiaddrs = [];
}
if (!networkOpts.discv5) {
networkOpts.discv5 = defaultDiscv5Options;
}
for (const enrOrStr of networkOpts.discv5.bootEnrs) {
const enr = typeof enrOrStr === "string" ? ENR.decodeTxt(enrOrStr) : enrOrStr;
const fullMultiAddr = await enr.getFullMultiaddr("tcp");
Expand Down
2 changes: 2 additions & 0 deletions packages/lodestar/src/network/peers/discover.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ export class PeerDiscovery {
await this.discv5.start();
this.discv5StartMs = Date.now();
this.discv5.on("discovered", this.onDiscovered);
// on start, dial the discv5.bootEnrs that has been added to the routing table
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have two choices here:

  1. only do this if network.connectToDiscv5Bootnodes is true
  2. remove the network.connectToDiscv5Bootnodes option entirely

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not supposed to connect to the Mainnet bootnodes, this has to be behind a cli flag and used only for local devnets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added it behind the network.connectToDiscv5Bootnodes cli flag

this.discv5.kadValues().forEach((enr) => this.onDiscovered(enr));
}

async stop(): Promise<void> {
Expand Down
3 changes: 3 additions & 0 deletions packages/validator/src/validator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ export class Validator {
/** Waits for genesis and genesis time */
static async initializeFromBeaconNode(opts: ValidatorOptions, signal?: AbortSignal): Promise<Validator> {
const {config} = opts.dbOps;
// TODO [DA] is there any objection to renaming api to client?
// it then becomes obvious this is a client that makes a request to an api
// instead of this being an api that is being made available
dadepo marked this conversation as resolved.
Show resolved Hide resolved
const api =
typeof opts.api === "string"
? getClient(config, {baseUrl: opts.api, timeoutMs: 12000, getAbortSignal: () => signal})
Expand Down