Merge remote-tracking branch 'upstream/master' into feat: draft imple…

…mentation of NEP-366 #7497

.buildkite/pipeline.yml

@@ -5,7 +5,11 @@ env:
 steps:
   - label: "protobuf backward compatibility"
     command: |
-      buf breaking --against ".git#ref=`git merge-base master HEAD`"
+      # We need to manually fetch the latest commit of master to
+      # correctly compute the commit at which we should diff the proto
+      # files. Apparently the pipeline doesn't do that for us.
+      git fetch origin
+      buf breaking --against ".git#ref=`git merge-base origin/master HEAD`"
 
     timeout: 30
     agents:
@@ -14,7 +18,8 @@ steps:
   - label: "cargo test integration-tests*"
     command: |
       source ~/.cargo/env && set -eux
-      cargo test --locked -p 'integration-tests'
+      cargo install cargo-nextest
+      cargo nextest run --locked -p 'integration-tests'
 
     timeout: 60
     agents:
@@ -24,7 +29,8 @@ steps:
   - label: "cargo test not integration-tests*"
     command: |
       source ~/.cargo/env && set -eux
-      cargo test --locked --workspace -p '*' --exclude 'integration-tests*'
+      cargo install cargo-nextest
+      cargo nextest run --locked --workspace -p '*' --exclude 'integration-tests*'
 
     timeout: 60
     agents:
@@ -34,7 +40,8 @@ steps:
   - label: "cargo test nightly integration-tests*"
     command: |
       source ~/.cargo/env && set -eux
-      cargo test --features nightly,test_features -p 'integration-tests'
+      cargo install cargo-nextest
+      cargo nextest run --features nightly,test_features -p 'integration-tests'
 
     timeout: 60
     agents:
@@ -44,7 +51,9 @@ steps:
   - label: "cargo test nightly not integration-tests*"
     command: |
       source ~/.cargo/env && set -eux
-      cargo test --workspace --features nightly,test_features,mock_node -p '*' --exclude 'integration-tests*'
+      cargo install cargo-nextest
+      cargo nextest run --workspace --features nightly,test_features,mock_node -p '*' --exclude 'integration-tests*'
+      cargo test --doc
 
     timeout: 60
     agents:
@@ -81,6 +90,8 @@ steps:
 
       ./scripts/formatting --check
 
+      ./scripts/run_clippy.sh
+
       rm target/rpc_errors_schema.json
       cargo check -p near-jsonrpc --features dump_errors_schema
       if ! git --no-pager diff --no-index chain/jsonrpc/res/rpc_errors_schema.json target/rpc_errors_schema.json; then

.cargo/config.toml

@@ -1,4 +1,7 @@
 [build]
 # We compile with `panic=abort`, so we need `-Cforce-unwind-tables=y`
 # to get a useful backtrace on panic.
-rustflags = ["-Ctarget-feature=+sse4.1,+sse4.2", "-Cforce-unwind-tables=y"]
+rustflags = ["-Cforce-unwind-tables=y"]
+
+[target.'cfg(target_arch = "x86_64")']
+rustflags = ["-Ctarget-feature=+sse4.1,+sse4.2", "-Cforce-unwind-tables=y"]

.config/nextest.toml

@@ -0,0 +1,2 @@
+[profile.default]
+slow-timeout = { period = "60s", terminate-after = 2, grace-period = "0s" }

CHANGELOG.md

@@ -4,31 +4,82 @@
 
 ### Protocol Changes
 
+### Non-protocol Changes
+
+## 1.31.0
+
+### Non-protocol Changes
+
+* Enable TIER1 network. Participants of the BFT consensus (block & chunk producers) now can establish direct TIER1 connections
+  between each other, which will optimize the communication latency and minimize the number of dropped chunks.
+  To configure this feature, see [advanced\_configuration/networking](./docs/advanced_configuration/networking.md).
+  [#8141](https://github.com/near/nearcore/pull/8141)
+  [#8085](https://github.com/near/nearcore/pull/8085)
+  [#7759](https://github.com/near/nearcore/pull/7759)
+* [Network] Started creating connections with larger nonces, that are periodically
+  refreshed Start creating connections (edges) with large nonces
+  [#7966](https://github.com/near/nearcore/pull/7966)
+* `/status` response has now two more fields: `node_public_key` and
+  `validator_public_key`.  The `node_key` field is now deprecated and should not
+  be used since it confusingly holds validator key.
+  [#7828](https://github.com/near/nearcore/pull/7828)
+* Added `near_node_protocol_upgrade_voting_start` Prometheus metric whose value
+  is timestamp when voting for the next protocol version starts.
+  [#7877](https://github.com/near/nearcore/pull/7877)
+* neard cmd can now verify proofs from JSON files.
+  [#7840](https://github.com/near/nearcore/pull/7840)
+* In storage configuration, the value `trie_cache_capacities` now is no longer
+  a hard limit but instead sets a memory consumption limit. For large trie nodes,
+  the limits are close to equivalent. For small values, there can now fit more
+  in the cache than previously.
+  [#7749](https://github.com/near/nearcore/pull/7749)
+* New options `store.trie_cache` and `store.view_trie_cache` in `config.json`
+  to set limits on the trie cache. Deprecates the never announced
+  `store.trie_cache_capacities` option which was mentioned in previous change.
+  [#7578](https://github.com/near/nearcore/pull/7578)
+* New option `store.background_migration_threads` in `config.json`. Defines
+  number of threads to execute background migrations of storage. Currently used
+  for flat storage migration. Set to 8 by default, can be reduced if it slows down
+  block processing too much or increased if you want to speed up migration.
+  [#8088](https://github.com/near/nearcore/pull/8088),
+* Tracing of work across actix workers within a process:
+  [#7866](https://github.com/near/nearcore/pull/7866),
+  [#7819](https://github.com/near/nearcore/pull/7819),
+  [#7773](https://github.com/near/nearcore/pull/7773).
+* Scope of collected tracing information can be configured at run-time:
+  [#7701](https://github.com/near/nearcore/pull/7701).
+* Attach node's `chain_id`, `node_id`, and `account_id` values to tracing
+  information: [#7711](https://github.com/near/nearcore/pull/7711).
+* Change exporter of tracing information from `opentelemetry-jaeger` to
+  `opentelemetry-otlp`: [#7563](https://github.com/near/nearcore/pull/7563).
+* Tracing of requests across processes:
+  [#8004](https://github.com/near/nearcore/pull/8004).
+* Gas profiles as displayed in the `EXPERIMENTAL_tx_status` are now more
+  detailed and give the gas cost per parameter.
+
+## 1.30.0
+
+### Protocol Changes
+
 * Stabilize `account_id_in_function_call_permission` feature: enforcing validity
   of account ids in function call permission.
-* Enable TIER1 peer discovery. Validator nodes are now exchanging the [public addresses
-  set in config](https://github.com/near/nearcore/blob/301fb493ea4f6d9b75d7dac7f2b52d00a1b2b709/chain/network/src/config_json.rs#L162).
-  The TIER1 connections support (direct connections between validators) based on
-  this discovery mechanism will be added soon.
+  [#7569](https://github.com/near/nearcore/pull/7569)
 
 ### Non-protocol Changes
 
 * `use_db_migration_snapshot` and `db_migration_snapshot_path` options are now
   deprecated.  If they are set in `config.json` the node will fail if migration
   needs to be performed.  Use `store.migration_snapshot` instead to configure
   the behaviour [#7486](https://github.com/near/nearcore/pull/7486)
-* `/status` response has now two more fields: `node_public_key` and
-  `validator_public_key`.  The `node_key` field is now deprecated and should not
-  be used since it confusingly holds validator key.
 * Added `near_peer_message_sent_by_type_bytes` and
   `near_peer_message_sent_by_type_total` Prometheus metrics measuring
   size and number of messages sent to peers.
+  [#7523](https://github.com/near/nearcore/pull/7523)
 * `near_peer_message_received_total` Prometheus metric is now deprecated.
   Instead of it aggregate `near_peer_message_received_by_type_total` metric.
   For example, to get total rate of received messages use
   `sum(rate(near_peer_message_received_by_type_total{...}[5m]))`.
-* Added `near_node_protocol_upgrade_voting_start` Prometheus metric whose value
-  is timestamp when voting for the next protocol version starts.
+  [#7548](https://github.com/near/nearcore/pull/7548)
 * Few changes to `view_state` JSON RPC query:
   - The requset has now an optional `include_proof` argument.  When set to
     `true`, response’s `proof` will be populated.
@@ -39,36 +90,15 @@
     sent even if proof has not been requested.  In the future the field will be
     skipped in those cases.  Clients should accept responses with this field
     missing (unless they set `include_proof`).
+    [#7603](https://github.com/near/nearcore/pull/7603)
 * Backtraces on panics are enabled by default, so you no longer need to set
   `RUST_BACKTRACE=1` environmental variable. To disable backtraces, set
-  `RUST_BACKTRACE=0`.
+  `RUST_BACKTRACE=0`. [#7562](https://github.com/near/nearcore/pull/7562)
 * Enable receipt prefetching by default. This feature makes receipt processing
   faster by parallelizing IO requests, which has been introduced in
   [#7590](https://github.com/near/nearcore/pull/7590) and enabled by default
   with [#7661](https://github.com/near/nearcore/pull/7661).
   Configurable in `config.json` using `store.enable_receipt_prefetching`.
-* neard cmd can now verify proofs from JSON files.
-* In storage configuration, the value `trie_cache_capacities` now is no longer
-  a hard limit but instead sets a memory consumption limit. For large trie nodes,
-  the limits are close to equivalent. For small values, there can now fit more
-  in the cache than previously.
-  [#7749](https://github.com/near/nearcore/pull/7749)
-* New options `store.trie_cache` and `store.view_trie_cache` in `config.json`
-  to set limits on the trie cache. Deprecates the never announced 
-  `store.trie_cache_capacities` option which was mentioned in previous change.
-  [#7578](https://github.com/near/nearcore/pull/7578)
-* Tracing of work across actix workers within a process:
-  [#7866](https://github.com/near/nearcore/pull/7866),
-  [#7819](https://github.com/near/nearcore/pull/7819),
-  [#7773](https://github.com/near/nearcore/pull/7773).
-* Scope of collected tracing information can be configured at run-time:
-  [#7701](https://github.com/near/nearcore/pull/7701).
-* Attach node's `chain_id`, `node_id`, and `account_id` values to tracing
-  information: [#7711](https://github.com/near/nearcore/pull/7711).
-* Change exporter of tracing information from `opentelemetry-jaeger` to
-  `opentelemetry-otlp`: [#7563](https://github.com/near/nearcore/pull/7563).
-* Tracing of requests across processes:
-  [#8004](https://github.com/near/nearcore/pull/8004).
 
 ## 1.29.0 [2022-08-15]
 

CONTRIBUTING.md

@@ -67,7 +67,7 @@ following steps when creating a PR:
 2. The branch can contain any number of commits.  When merged, all commits will
    be squashed into a single commit.
 3. The changes should be thoroughly tested.  Please refer to [this
-   document](https://github.com/nearprotocol/nearcore/wiki/Writing-tests-for-nearcore)
+   document](https://github.com/near/nearcore/blob/master/docs/practices/testing/README.md)
    for our testing guidelines and an overview of the testing infrastructure.
 4. When ready, send a pull request against the `master` branch of the `nearcore`
    repository.
@@ -117,6 +117,10 @@ who belongs either to the super owners or the normal owners group.
 well.  They will review your tests, and make sure that they can convince
 themselves the test coverage is adequate before they even look into the
 change, so make sure you tested all the corner cases.
+- it is normal to sometimes require multiple rounds of reviews to get a PR
+  merged.  If your PR received some feedback from a reviewer, use the [github
+  UI](https://stackoverflow.com/questions/40893008/how-to-resume-review-process-after-updating-pull-request-at-github)
+  to re-request a review.
 
 The author is also free to directly request reviews from specific persons
 [through the github