Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test related documentation issues #10168

Merged
merged 1 commit into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions docs/practices/testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,19 +36,18 @@ It requires nextest harness which can be installed by running `cargo install car
such tests is to enable all the features by passing `--all-features` to
`cargo nextest run`, e.g:

`cargo nextest run --package near-client --test cross_shard_tx
tests::test_cross_shard_tx --all-features`
`cargo nextest run --package near-client -E 'test(=tests::cross_shard_tx::test_cross_shard_tx)' --all-features`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue 2: Non-Existent Test Target Documentation


3. **Python tests:** We have an infrastructure to spin up nodes, both locally and
remotely, in python, and interact with them using RPC. The infrastructure and
the tests are located in the `pytest` folder. The infrastructure is relatively
straightforward, see for example `block_production.py`
[here](https://github.com/nearprotocol/nearcore/blob/master/pytest/tests/sanity/block_production.py).
[here](https://github.com/near/nearcore/blob/master/pytest/tests/sanity/block_production.py).
See the `Test infrastructure` section below for details.

Expensive and python tests are not part of CI, and are run by a custom nightly
runner. The results of the latest runs are available
[here](http://nightly.neartest.com/). Today, test runs launch approximately
[here](https://nayduck.near.org/#/). Today, test runs launch approximately
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue 1: Incorrect URL for Runner Test Results

every 5-6 hours. For the latest results look at the **second** run, since the
first one has some tests still scheduled to run.

Expand All @@ -71,7 +70,7 @@ predefined timeout.

For the most basic example of using this infrastructure see `produce_two_blocks`
in
[`tests/process_blocks.rs`](https://github.com/nearprotocol/nearcore/blob/master/chain/client/tests/process_blocks.rs).
[`tests/process_blocks.rs`](https://github.com/near/nearcore/blob/master/chain/client/src/tests/process_blocks.rs).

1. The callback (`Box::new(move |msg, _ctx, _| { ...`) is what is executed
whenever the client sends a message. The return value of the callback is sent
Expand All @@ -84,13 +83,13 @@ in

For an example of a test that launches multiple nodes, see
`chunks_produced_and_distributed_common` in
[tests/chunks_management.rs](https://github.com/nearprotocol/nearcore/blob/master/chain/client/tests/chunks_management.rs).
[tests/chunks_management.rs](https://github.com/near/nearcore/blob/master/chain/client/src/tests/chunks_management.rs).
The `setup_mock_all_validators` function is the key piece of infrastructure here.

## Runtime

Tests for Runtime are listed in
[tests/test_cases_runtime.rs](https://github.com/near/nearcore/blob/master/tests/test_cases_runtime.rs).
[tests/test_cases_runtime.rs](https://github.com/near/nearcore/blob/master/integration-tests/src/tests/standard_cases/runtime.rs).

To run a test, usually, a mock `RuntimeNode` is created via `create_runtime_node()`.
In its constructor, the `Runtime` is created in the
Expand All @@ -114,16 +113,16 @@ make sure to build new components sufficiently abstract so that they can be test
without relying on other components.

For example, see tests for doomslug
[here](https://github.com/nearprotocol/nearcore/blob/master/chain/chain/tests/doomslug.rs),
[here](https://github.com/near/nearcore/blob/master/chain/chain/src/tests/doomslug.rs),
for network cache
[here](https://github.com/nearprotocol/nearcore/blob/master/chain/network/tests/cache_edges.rs),
[here](https://github.com/near/nearcore/blob/master/chain/network/src/routing/edge_cache/tests.rs),
or for promises in runtime
[here](https://github.com/nearprotocol/nearcore/blob/master/runtime/near-vm-logic/tests/test_promises.rs).
[here](https://github.com/near/nearcore/blob/master/runtime/near-vm-runner/src/logic/tests/promises.rs).

## Python tests

See
[this page](https://github.com/nearprotocol/nearcore/wiki/Writing-integration-tests-for-nearcore)
[this page](python_tests.md)
for detailed coverage of how to write a python test.

We have a python library that allows one to create and run python tests.
Expand Down
8 changes: 4 additions & 4 deletions docs/practices/testing/python_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ want to run tests against a release build, you can create a file with the
following config:

```json
{"local": True, "near_root": "../target/release/"}
{"local": true, "near_root": "../target/release/"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue 3: Misguiding Python Tests Documentation

```

and run the test with the following command:
Expand Down Expand Up @@ -116,7 +116,7 @@ Note that `start_cluster` spins up all the nodes right away. Some tests (e.g.
tests that test syncing) might want to configure the nodes but delay their
start. In such a case you will initialize the cluster by calling
`init_cluster` and will run the nodes manually, for example, see
[`state_sync.py`](https://github.com/nearprotocol/nearcore/blob/master/pytest/tests/sanity/state_sync.py)
[`state_sync.py`](https://github.com/near/nearcore/blob/master/pytest/tests/sanity/state_sync.py)

## Connecting to a mocknet

Expand Down Expand Up @@ -234,7 +234,7 @@ if want_async:
```

See
[rpc_tx_forwarding.py](https://github.com/nearprotocol/nearcore/blob/master/pytest/tests/sanity/rpc_tx_forwarding.py)
[rpc_tx_forwarding.py](https://github.com/near/nearcore/blob/master/pytest/tests/sanity/rpc_tx_forwarding.py)
for an example of signing and submitting a transaction.

## Adversarial behavior
Expand Down Expand Up @@ -269,6 +269,6 @@ See the tests that match `tests/sanity/proxy_*.py` for examples.

We always welcome new tests, especially python tests that use the above
infrastructure. We have a list of test requests
[here](https://github.com/nearprotocol/nearcore/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+test%22+),
[here](https://github.com/nearprotocol/nearcore/issues?q=is%3Aissue+is%3Aopen+label%3A%22A-testing%22),
but also welcome any other tests that test aspects of the network we haven't
thought about.
Loading