diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 86ff2ce..061a879 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -4,6 +4,7 @@ - [Getting Started](./getting_started.md) - [Hardware Requirements](./getting-started/hw-requirements.md) + - [Tips for faster syncing](./getting-started/tips.md) - [Software Requirements](./getting-started/sw-requirements.md) - [Installation](installation.md) @@ -23,13 +24,14 @@ - [Advanced Usage](./advanced.md) - [Configuring Erigon](./advanced/configuring.md) - [Consensus Layer](./advanced/consensus_layer.md) + - [Caplin](./advanced/caplin.md) - [Options](./advanced/options.md) - - [Caplin]()(./advanced/caplin.md) - [RPC Daemon](./advanced/rpc_daemon.md) - [TxPool](./advanced/txpool.md) - [Sentry](./advanced/sentry.md) - [Downloader](./advanced/downloader.md) - [TLS Authentication](./advanced/tls-authentication.md) + - [Perfomance Tricks](./advanced/performance.md) - [Diagnostic Tool](./diagnostic/introduction.md) - [Installation](./diagnostic/installation.md) diff --git a/src/advanced/caplin.md b/src/advanced/caplin.md new file mode 100644 index 0000000..13e7c87 --- /dev/null +++ b/src/advanced/caplin.md @@ -0,0 +1 @@ +# Caplin \ No newline at end of file diff --git a/src/advanced/performance.md b/src/advanced/performance.md new file mode 100644 index 0000000..122fc8d --- /dev/null +++ b/src/advanced/performance.md @@ -0,0 +1,43 @@ +# Performance Tricks + + +These instructions are designed to improve the performance of Erigon 3, particularly for synchronization and memory management, on cloud drives and other systems with specific performance characteristics. + + +## Increase Sync Speed + +* Set `--sync.loop.block.limit=10_000` and `--batchSize=2g` to speed up the synchronization process. +```bash +--sync.loop.block.limit=10_000 --batchSize=2g +``` + +## Optimize for Cloud Drives + +* Set `SNAPSHOT_MADV_RND=false` to enable the operating system's cache prefetching for better performance on cloud drives with good throughput but bad latency. +```bash +SNAPSHOT_MADV_RND=false +``` + +## Lock Latest State in RAM + +* Use `vmtouch -vdlw /mnt/erigon/snapshots/domain/*bt` to lock the latest state in RAM, preventing it from being evicted due to high historical RPC traffic. +```bash +vmtouch -vdlw /mnt/erigon/snapshots/domain/*bt +``` + +* Run `ls /mnt/erigon/snapshots/domain/*.kv | parallel vmtouch -vdlw` to apply the same locking to all relevant files. + +## Handle Memory Allocation Issues + +* If you encounter issues with memory allocation, run the following to flush any pending write operations and free up memory: +```bash +sync && sudo sysctl vm.drop_caches=3 +``` + +* Alternatively, set: + +```bash +echo 1 > /proc/sys/vm/compact_memory +``` + +to help with memory allocation. diff --git a/src/basic/networks.md b/src/basic/networks.md index abc259f..5a40388 100644 --- a/src/basic/networks.md +++ b/src/basic/networks.md @@ -1,17 +1,19 @@ # Supported Networks -Use the flag `--chain=` to sync one of the below supported network, for example: + +Utilize the flag `--chain=` to synchronize with one of the supported networks, for example: ```bash ./build/bin/erigon --chain=holesky ``` -The default flag is `chain=mainnet` which runs Erigon 3 on the Ethereum mainnet. +The default flag is `--chain=mainnet`, which enables Erigon 3 to operate on the Ethereum mainnet. + # Mainnets -|Chain | Tag| ChainId| -|----|----|----| +|Chain | Tag| ChainId| +|---- |----|----| |Ethereum| mainnet| 1| |Polygon| bor-mainnet| 137| |Gnosis| gnosis| 100| @@ -28,11 +30,9 @@ The default flag is `chain=mainnet` which runs Erigon 3 on the Ethereum mainnet. ## Polygon testnets -|Chain | Tag| ChainId| +|Chain | Tag| ChainId| |----------|-------|-----------| -|Polygon Mumbai | mumbai| 80001| -|Polygon Amoy | amoy| 80002| -|Polygon BorDevnet| bor-devnet| null| +|Polygon Amoy | amoy| 80002| ## Gnosis Testnets @@ -41,10 +41,3 @@ The default flag is `chain=mainnet` which runs Erigon 3 on the Ethereum mainnet. |----------|-------|-----------| |Chiado | chiado| 10200| -## Development testnet - -|Chain | Tag| ChainId| -|----------|-------|-----------| -|Dev| dev| null| - - diff --git a/src/basic/ports.md b/src/basic/ports.md index a1127c6..d2bc10a 100644 --- a/src/basic/ports.md +++ b/src/basic/ports.md @@ -1 +1,61 @@ # Default Ports and Firewalls + +Erigon Ports +------------- + +| Component | Port | Protocol | Purpose | Should Expose | +| --------- | ---- | -------- | ------- | -------------- | +| Engine | 9090 | TCP | gRPC Server | Private | +| Engine | 42069 | TCP & UDP | Snap sync (Bittorrent) | Public | +| Engine | 8551 | TCP | Engine API (JWT auth) | Private | +| Sentry | 30303 | TCP & UDP | eth/68 peering | Public | +| Sentry | 30304 | TCP & UDP | eth/67 peering | Public | +| Sentry | 9091 | TCP | incoming gRPC Connections | Private | +| RPCdaemon | 8545 | TCP | HTTP & WebSockets & GraphQL | Private | +| Diagnostics | 8080 | TCP & UDP| Diagnostic Tool | Private | + +Caplin ports +------------ + +| Component | Port | Protocol | Purpose | Should Expose | +| --------- | ---- | -------- | ------- | -------------- | +| Sentinel | 4000 | UDP | Peering | Public | +| Sentinel | 4001 | TCP | Peering | Public | + +Shared ports +------------ + +| Component | Port | Protocol | Purpose | Should Expose | +| --------- | ---- | -------- | ------- | -------------- | +| All | 6060 | TCP | pprof | Private | +| All | 6060 | TCP | metrics | Private | + +Optional flags can be enabled that enable pprof or metrics (or both) - however, they both run on 6060 by default, so you'll have to change one if you want to run both at the same time. + +Other ports +------------ + +* gRPC ports: 9092 consensus engine, 9093 snapshot downloader, 9094 TxPool + +Hetzner firewall rules +---------------------- + +* `0.0.0.0/8`: "This" Network (RFC 1122, Section 3.2.1.3) +* `10.0.0.0/8`: Private-Use Networks (RFC 1918) +* `100.64.0.0/10`: Carrier-Grade NAT (CGN) (RFC 6598, Section 7) +* `127.16.0.0/12`: Private-Use Networks (RFC 1918) +* `169.254.0.0/16`: Link Local (RFC 3927) +* `172.16.0.0/12`: Private-Use Networks (RFC 1918) +* `192.0.0.0/24`: IETF Protocol Assignments (RFC 5736) +* `192.0.2.0/24`: TEST-NET-1 (RFC 5737) +* `192.88.99.0/24`: 6to4 Relay Anycast (RFC 3068) +* `192.168.0.0/16`: Private-Use Networks (RFC 1918) +* `198.18.0.0/15`: Network Interconnect Device Benchmark Testing (RFC 2544) +* `198.51.100.0/24`: TEST-NET-2 (RFC 5737) +* `203.0.113.0/24`: TEST-NET-3 (RFC 5737) +* `224.0.0.0/4`: Multicast (RFC 3171) +* `240.0.0.0/4`: Reserved for Future Use (RFC 1112, Section 4) +* `255.255.255.255/32`: Limited Broadcast (RFC 919, Section 7) +* `255.255.255.255/32`: RFC 922, Section 7 + +Same in [IpTables](https://ethereum.stackexchange.com/questions/6386/how-to-prevent-being-blacklisted-for-running-an-ethereum-client/13068#13068) syntax. \ No newline at end of file diff --git a/src/basic/wallet b/src/basic/wallet index c4ff9ec..0c0dc51 100644 --- a/src/basic/wallet +++ b/src/basic/wallet @@ -22,8 +22,8 @@ To configure your local Metamask wallet (browser extension): * Click on the **network selector button**. This will display a list of networks to which you're already connected * Click **Add network** * A new browser tab will open, displaying various fields to fill out. Complete the fields with the proper information, in this example for Ethereum network: - * **Network Name**: Erigon (or any name of your choice) - * **Chain ID**: 1 for chain ID parameter see [Supported Networks](/networks.md) + * **Network Name**: `Ethereum on E3` (or any name of your choice) + * **Chain ID**: 1 for chain ID parameter see [Supported Networks](/basic/networks.md) * **New RPC URL**: http://127.0.0.1:8545 **Currency Symbol**: ETH **Block Explorer URL**: https://www.etherscan.io (or any explorer of your choice) diff --git a/src/getting-started/hw-requirements.md b/src/getting-started/hw-requirements.md index ca7d9f2..e185643 100644 --- a/src/getting-started/hw-requirements.md +++ b/src/getting-started/hw-requirements.md @@ -29,8 +29,3 @@ Please refer to [disk space required](/basic/disk-space.md) for details. To ensu * **Linux**: kernel version > v4 -# Tips for Faster Sync​ - -* **Optimize for Low Latency**: Use a machine with low latency (not throughput) disk and RAM for faster initial sync. -* **Memory Optimized Nodes**: Consider using memory-optimized nodes for faster sync, such as AWS EC2 r5 or r6 series instances. - diff --git a/src/getting-started/sw-requirements.md b/src/getting-started/sw-requirements.md index 4ddbc75..ed16845 100644 --- a/src/getting-started/sw-requirements.md +++ b/src/getting-started/sw-requirements.md @@ -23,13 +23,11 @@ Git is a tool that helps download and manage the Erigon source code. To install ### Go Programming Language -Go (also called Golang) version 1.21 or newer is used to write part of Erigon. +Erigon utilizes Go (also known as Golang) version 1.22 or newer for part of its development. It is recommended to have a fresh Go installation. If you have an older version, consider deleting the /usr/local/go folder (you may need to use sudo) and re-extract the new version in its place. -It is recommended to have a new Go installation. In case you have a previous version delete the /usr/local/go folder (you will probably need to usesudo), and re-extract the new version in its place. +To install the latest Go version, visit the official documentation at [https://golang.org/doc/install](https://golang.org/doc/install). -Visit [https://golang.org/doc/install](https://golang.org/doc/install) for installation instructions. - -For Linux copy-paste this into your terminal: +For Linux users, use the following command in your terminal: ```bash sudo apt-get update diff --git a/src/getting-started/tips.md b/src/getting-started/tips.md new file mode 100644 index 0000000..8d51c7c --- /dev/null +++ b/src/getting-started/tips.md @@ -0,0 +1,4 @@ +# Tips for faster syncing + +* **Optimize for Low Latency**: Use a machine with low latency (not throughput) disk and RAM for faster initial sync. +* **Memory Optimized Nodes**: Consider using memory-optimized nodes for faster sync, such as *AWS EC2 r5* or *r6* series instances. \ No newline at end of file diff --git a/src/welcome.md b/src/welcome.md index a0cdd01..5c991ce 100644 --- a/src/welcome.md +++ b/src/welcome.md @@ -14,18 +14,31 @@ Erigon is an efficiency frontier implementation of Ethereum, designed to provide Erigon offers several features that make it a good option for a node application such as efficient state storage through the use of a key-value database and faster initial synchronisation. -It also offers a separate JSON RPC daemon, that can connect to both local and remote databases. For read-only calls, this RPC daemon does not need to run on the same system as the main Erigon binary, and can even run from a database snapshot. +Built with modularity in mind, it also offers separate components such as the JSON RPC daemon, that can connect to both local and remote databases. For read-only calls, this RPC daemon does not need to run on the same system as the main Erigon binary, and can even run from a database snapshot. Erigon 3 is a major update that introduces several significant changes, improvements, and optimizations. Some of the key features and differences include: + +**Sync Improvements** + +* **Sync from scratch** doesn't require re-executing **all history**. Instead, it leverages **snapshots** to download the latest state and its history. +* **ExecutionStage** now includes multiple E2 stages, such as ``stage_hash_state``, ``stage_trie``, ``stage_log_index``, ``stage_history_index``, and ``stage_trace_index``, for faster execution. +* **E3 can execute a single historical transaction** without executing its corresponding block, thanks to its **transaction-granularity indexing system**, which differs from a block-granularity approach. +* **E3 doesn't store Logs** (aka Receipts) - it always re-executes historical transactions (but it's cheaper than in E2 - see point above). Known performance issues: #10747 +* **--sync.loop.block.limit** is enabled by default, with a default value of 5,000. To increase sync speed on good hardware, set ``--sync.loop.block.limit=10_000 --batchSize=2g``. + +**Storage and Performance** + +* **datadir/chaindata is small now** - to prevent its growth: we recommend setting ``--batchSize <= 2G``. It's also fine to remove the chaindata directory. +* Symlink or mount the latest state to a fast drive and history to a cheaper drive. + +**Node Configuration** + +* **Archive Node is default**. Full Node: ``--prune.mode=full``, Minimal Node (EIP-4444): ``--prune.mode=minimal`` + +**Other Features** + * **New User Guide**: A dedicated user guide for Erigon 3 provides detailed information on how to set up and configure your node. -* **Higher RAM Requirement**: Erigon 3 requires at least 64GB of RAM (which may be reduced in future releases). -* **Golang 1.21**: The new version is built with Golang 1.21, which brings various improvements and enhancements. -* **Sync from Scratch**: Syncing from scratch doesn't require re-executing all history; instead, it uses snapshots to download the latest state and its history. -* **ExecutionStage**: Erigon 3 includes multiple stages in ExecutionStage (e.g., stage_hash_state, stage_trie) for faster execution. -* **Transaction-Level Execution**: Erigon 3 can execute individual historical transactions without executing their entire block, thanks to transaction-granularity indices. -Log Storage Removal: Logs (Receipts) are no longer stored in the datadir; instead, they are re-executed when needed. -* **Default Archive Node**: Erigon 3 is now an archive node by default, with pruning options for [Full Nodes](./basic/node.md#full-node) and [Minimal Nodes (EIP-4444)](./basic/node.md#minimal-node). -* **Improved Performance**: The new version offers faster performance, reduced startup time, and lower read-IO. +* **Higher RAM Requirement**: Erigon 3 requires at least 64GB of RAM (which may be ## Release Process @@ -37,6 +50,4 @@ Erigon 3 also introduces changes to the release process, including: ## Known Issues While Erigon 3 offers many improvements over its predecessor, there are still some known issues and limitations: -- `Trace_callMany` support for multiple transactions: #11798 -- Windows binaries are not yet published, but will be available once the necessary work is completed. -- Users may encounter performance issues on chain-tip; `rm -rf chaindata` can help resolve these issues. \ No newline at end of file +- don't [rm -rf] downloader - it will cause re-downloading of files: #10976