Merge branch 'master' into https-config

.travis.yml

@@ -11,10 +11,11 @@ before_script:
   - pip install pymdown-extensions
   - pip install mkdocs-material
   - pip install MarkdownHighlight
+  - pip install mkdocs-redirects
 script:
   - |
     if [ "${TRAVIS_PULL_REQUEST}" != "false" ]; then
-      mkdocs build
+      mkdocs build --strict
     elif [ "${TRAVIS_PULL_REQUEST}" = "false" ] && [ "${TRAVIS_BRANCH}" = "master" ]; then
       openssl aes-256-cbc -K $encrypted_5f828affe522_key -iv $encrypted_5f828affe522_iv -in deploy-key.enc -out deploy-key -d
       chmod 600 deploy-key

docs/commands/command-line-interface.md

@@ -1,22 +1,37 @@
 # Command Line Interface
 
 ### --account_create --wallet=`<wallet>`
-Insert next deterministic key in to `<wallet>`
+Insert next deterministic key into `<wallet>`
 
 ### --account_get --key=`<key>`
 Get account number for the `<key>`
 
 ### --account_key --account=`<account>`
 Get the public key for `<account>`
 
-### --clear_send_ids   
+### --clear_send_ids
 Remove all send IDs from the database (dangerous: not intended for production use)
 
+### --compare_rep_weights
+_version 21.0+_  
+Displays a summarized comparison between the hardcoded bootstrap weights and representative weights from the ledger. Full comparison is output to logs. Optional [`--data_path`](#-data_pathpath).
+
+* Differences between total weights (`hardcoded weight` and `ledger weight`) are due to unreceived (pending) blocks
+* `mismatched`:
+    * `samples`: the number of mismatched samples is equal to the number of hardcoded weights, even those with zero mismatch
+    * `total`: sum of the absolute difference between individual samples from hardcoded and ledger weights
+    * `mean`: `total` divided by `samples`
+    * `sigma`: from the samples, a distribution $N(\mu, \sigma)$ is obtained
+* `outliers`: mismatch samples above $\mu + \sigma$, for potential inspection
+* `newcomers`: large voting weights found in the ledger but not hardcoded, for potential inspection
+
 ### --config key=value
+_version 20.0+_  
 Pass node configuration values. This takes precedence over any values in the configuration file. This option can be repeated multiple times.
 
 ### --confirmation_height_clear
-_version 19.0+_ Sets the confirmation heights of all accounts to 0. Optional `--account` to only reset a single account.
+_version 19.0+_  
+Sets the confirmation heights of all accounts to 0. Optional `--account` to only reset a single account. Do not use while the node is running.
 
 ### --daemon
 Start node daemon. Since version 19.0, network and path will be output, similar to:
@@ -27,78 +42,13 @@ Path: /home/USER/NanoTest
 ```
 
 ### --data_path=`<path>` 
-Use the supplied `<path>` as the data directory
-
-### --debug_account_count	
-Display the number of accounts
-
-### --debug_account_versions	
-_version 20.0+_ Display the total counts of each version for all accounts (including unpocketed)
-
-### --debug_block_count
-Display the number of blocks
-
-### --debug_bootstrap_generate
-Generate bootstrap sequence of blocks
-
-### --debug_cemented_block_count
-_version 19.0+_ Display the number of cemented blocks (blocks which are under the confirmation height of their accounts)
-
-### --debug_dump_frontier_unchecked_dependents
-_version 19.0+_ Dump frontiers which have matching unchecked keys
-
-### --debug_dump_online_weight 
-List online weights table and current online_weights value
-
-### --debug_dump_representatives
-List representatives and weights
-
-### --debug_mass_activity
-Generates fake debug activity
-
-### --debug_output_last_backtrace_dump
-Output the stacktrace stored after a node crash.
-
-### --debug_profile_bootstrap
-Profile simulated bootstrap process
-
-### --debug_profile_generate
-Profile work generation  
-Optional `--pow_sleep_interval` in version 19.0+ which sets an amount to sleep (in nanoseconds) between batches of POW calculations when using the CPU.
-
-### --debug_profile_validate
-Profile work validation
-
-### --debug_profile_kdf
-Profile kdf function
-
-### --debug_profile_sign
-Profile signature generation
-
-### --debug_profile_votes
-Profile vote verification
-
-### --debug_stacktrace
-_version 20.0+_
-Prints a stacktrace example, useful to verify that it includes the desired information, such as files, function names and line numbers
-
-### --debug_validate_blocks
-_version 19.0+_
-Validate blocks in the ledger, includes checks for confirmation height
-
-### --debug_verify_profile
-Profile signature verification
-
-### --debug_xorshift_profile
-[Disabled] Profile xorshift algorithms
-
-### --debug_opencl --platform=`<platform>` --device=`<device>` --threads=`<threads>`
-_[Draft]_ Profile OpenCL work generation for `<device>` on `<platform>` using `<threads>` count. To retrieve available platforms & devices run --diagnostics
+Use the supplied `<path>` as the data directory.
 
 ### --diagnostics
 Run internal diagnostics and validate existing config file (or create default config file if it doesn't exist)
 
 ### --generate_config node|rpc
+_version 20.0+_  
 Write configuration to stdout, populated with commented-out defaults suitable for this system. Pass the configuration type, `node` or `rpc`.
 If `--use_defaults` is passed, the generated config will not have values commented-out. This is not recommended except for testing and debugging.
 
@@ -114,23 +64,36 @@ Generates a adhoc random keypair and prints it to stdout
 Derive public key and account number from `<key>`
 
 ### --network
+_version 19.0+_  
 Allows selection of a different network at runtime. Values `live`, `beta` and `test` supported.
 
 ### --online_weight_clear
+_version 18.0+_  
 Clear record history for long term online weight trending
 
 ### --peer_clear
+_version 18.0+_  
 Clear cached peers
 
+### --rebuild_database
+_version 21.0+_  
+Rebuild LMDB database with `--vacuum` for best compaction. Requires approximately `data.ldb size * 2` free space on disk.
+
 ### --snapshot
-Compact database and create snapshot, functions similar to vacuum but does not replace the existing database. Optional `--unchecked_clear`, `--clear_send_ids`, `--online_weight_clear`, `--peer_clear`
-Optional `--confirmation_height_clear` in version 19.0+
+Compact database and create snapshot, functions similar to vacuum but does not replace the existing database. Optional `--unchecked_clear`, `--clear_send_ids`, `--online_weight_clear`, `--peer_clear`.
+Optional `--confirmation_height_clear` in version 19.0+.
 
 ### --unchecked_clear
 Clear unchecked blocks
 
 ### --vacuum
-Compact database. If data_path is missing, the database in data directory is compacted. Optional `--unchecked_clear`, `--clear_send_ids`, `--online_weight_clear`, `--peer_clear`
+Compact database. If data_path is missing, the database in data directory is compacted. Optional `--unchecked_clear`, `--clear_send_ids`, `--online_weight_clear`, `--peer_clear`.
+Optional `--confirmation_height_clear` in version 19.0+.
+Optional `--rebuild_database` in version 21.0+. Requires approximately `data.ldb size * 2` free space on disk.
+
+### --validate_blocks
+_version 21.0+_ (_version 19.0+_ as `--debug_validate_blocks`)  
+Validate blocks in the ledger, includes checks for confirmation height. Optional `--threads` for multithreaded validation in version 21.0+. Multithreaded validation can limit other host operations with high I/O & CPU usage.
 
 ### --version    
 Prints out version
@@ -149,7 +112,10 @@ Changes seed for `<wallet>` to `<seed>`.  Note the legacy `--key` option can sti
 
 ### --wallet_decrypt_unsafe --wallet=`<wallet>` --password=`<password>`
 Decrypts `<wallet>` using `<password>`  
-**!!THIS WILL PRINT YOUR PRIVATE KEY AND SEED TO STDOUT!!**  
+
+!!! danger
+	**USE WITH CAUTION: THIS WILL PRINT YOUR PRIVATE KEY AND SEED TO STDOUT**
+
 If you didn't set password yet, use --wallet_decrypt_unsafe --wallet=`<wallet>`
 
 ### --wallet_destroy --wallet=`<wallet>`
@@ -170,10 +136,16 @@ Prints default representative for `<wallet>`
 ### --wallet_representative_set --wallet=`<wallet>` --account=`<account>`
 Set `<account>` as default representative for `<wallet>`
 
+
 ## Launch options
 When initially starting the nano_node or nano_wallet as a service the following launch options are available.
 
-NOTE: These options are only for developer use so please understand the impacts before use.
+!!! note "Intended for developer use"
+	These options are only for developer use so please understand the impacts before use.
+
+### --allow_bootstrap_peers_duplicates
+_version 21.0+_  
+Allow multiple connections to the same peer in bootstrap attempts
 
 ### --block_processor_batch_size
 Increase block processor transaction batch write size, default 0 (limited by config block_processor_batch_max_time), 256k for fast_bootstrap
@@ -200,18 +172,133 @@ Turn off use of wallet-based bootstrap
 Turn off listener on the bootstrap network so incoming TCP (bootstrap) connections are rejected. **Note:** this does not impact TCP traffic for the live network.
 
 ### --disable_tcp_realtime
-_version 19.0+_
+_version 19.0+_  
 Turn off use of TCP live network (TCP for bootstrap will remain available)
 
-### --disable_udp
-_version 19.0+_
-Turn off use of UDP live network
-
 ### --disable_unchecked_cleanup
 Prevent periodic cleaning of unchecked table
 
 ### --disable_unchecked_drop
 Prevent drop of all unchecked entries at node/wallet start
 
+### --disable_providing_telemetry_metrics
+_version 21.0+_  
+Do not provide any telemetry data to nodes requesting it. Responses are still made to requests, but they will have an empty payload.
+
+### --disable_block_processor_unchecked_deletion
+_version 21.0+_  
+Disable deletion of unchecked blocks after processing.
+
+### --enable_udp
+_version 21.0+_  
+Turn on use of the UDP live network.
+
 ### --fast_bootstrap
-Increase bootstrap processor limits to allow more blocks before hitting full state and verify/write more per database call. Also disable deletion of processed unchecked blocks
+Increase bootstrap processor limits to allow more blocks before hitting full state and verify/write more per database call. Also disable deletion of processed unchecked blocks.
+
+### --inactive_votes_cache_size
+_version 21.0+_  
+Increase cached votes without active elections size, default 16384
+
+### --vote_processor_capacity
+_version 21.0+_  
+Vote processor queue size before dropping votes, default 144k
+
+## Debug commands
+
+### --debug_account_count
+Display the number of accounts
+
+### --debug_account_versions
+_version 20.0+_  
+Display the total counts of each version for all accounts (including unpocketed)
+
+### --debug_block_count
+Display the number of blocks
+
+### --debug_bootstrap_generate
+Generate bootstrap sequence of blocks
+
+### --debug_cemented_block_count
+_version 19.0+_  
+Display the number of cemented blocks (blocks which are under the confirmation height of their accounts)
+
+### --debug_dump_frontier_unchecked_dependents
+_version 19.0+_  
+Dump frontiers which have matching unchecked keys
+
+### --debug_dump_online_weight
+List online weights table and current online_weights value
+
+### --debug_dump_representatives
+List representatives and weights
+
+### --debug_generate_crash_report
+_version 21.0+_
+After a node crash on linux, this command consumes the dump files generated from that crash and produces a "nano_node_crash_report.txt" file. Requires `addr2line` to be installed on the system. See the [troubleshooting guide](/running-a-node/troubleshooting/#what-to-do-if-the-node-crashes-linux) for more information.
+
+### --debug_opencl
+Profile OpenCL work generation for (optional) `--device=<device>` on `--device=<platform>` using `--threads=<threads>` count. To retrieve available platforms & devices run [--diagnostics](#-diagnostics). 
+
+### --debug_output_last_backtrace_dump
+_version 19.0+_  
+Output the stacktrace stored after a node crash. 
+
+Optionals `--difficulty` and `--multiplier` (only the latter is used if both given) in version 21.0+ to set the work generation threshold.
+
+### --debug_profile_bootstrap
+Profile simulated bootstrap process
+
+### --debug_profile_generate
+Profile work generation  
+Optional `--pow_sleep_interval` in version 19.0+ which sets an amount to sleep (in nanoseconds) between batches of POW calculations when using the CPU.  
+Optionals `--difficulty` and `--multiplier` (only the latter is used if both given) in version 21.0+ to set the work generation threshold.
+
+### --debug_profile_validate
+Profile work validation
+
+### --debug_profile_kdf
+Profile kdf function
+
+### --debug_profile_sign
+Profile signature generation
+
+### --debug_profile_votes
+Profile vote verification
+
+### --debug_profile_frontiers_confirmation
+_version 21.0+_
+Profile frontiers confirmation speed
+
+### --debug_stacktrace
+_version 20.0+_  
+Prints a stacktrace example, useful to verify that it includes the desired information, such as files, function names and line numbers
+
+### --debug_validate_blocks
+Alias to [`--validate_blocks`](#-validate_blocks)
+
+### --debug_verify_profile
+Profile signature verification
+
+### --debug_xorshift_profile
+[Disabled] Profile xorshift algorithms
+
+## Deprecated commands
+
+### Debug
+
+##### --debug_mass_activity (Deprecated)
+Generates fake debug activity. Deprecated in _v21+_, can use `slow_test --gtest_filter=system.generate_mass_activity` instead.
+
+### Launch options
+
+##### --batch_size (Deprecated)
+_version 18.0+_  
+Increase sideband upgrade batch size (default 512). Deprecated in _v21_ and will be removed in _v22_ as it will not support upgrades from v18 nodes and earlier.
+
+##### --disable_udp (Deprecated)
+_version 21.0+_  
+This option has been deprecated and will be removed in future versions. It has no effect because it is now the default.
+
+_version 19.0+_  
+Turn off use of UDP live network