A black/grey box suite that tests the functionality of the 0Chain network as an end user via the CLI tools.
The tests require a full 0Chain network to be deployed and running in a healthy state.
The System Tests Pipeline can deploy a new 0Chain network with a custom set of docker images then run tests:
[Click to show screenshot]
In this mode, do not supply the network URL. Supply the docker images you wish to deploy
You can view a list available 0chain docker images at Docker Hub, or build your own by running the docker build pipeline in the repo of your feature branch.
0Chain will automatically be deployed to a free test slot at dev-[1-5].devnet-0chain.net.
You can view the network URL of deployment by checking the "VIEW TEST CONFIGURATION" step of the pipeline.
[Click to show screenshot]
If tests fail, the network will stay available for debugging purposes, however uptime is not guaranteed as the network may be overridden by another test run.
Run system tests to unblock merging code changes spanning multiple repos protected by automatic system tests
There may arise a scenario when PRs will fail against default system tests. Consider this, you raise a PR on zboxcli, blobber and 0chain repos simultaneously that only work with new code on all three of them. In this case the PRs raised on all 3 of them will fail system tests because the PR raised on zboxcli will have the new code in that zboxcli branch, but not the required blobber or 0chain code, and so on for the other 2 repos.
In such a case, go to actions tab on any of the repos, find the 0Chain System Tests workflow (like here), input all the custom branches and run it manually. After the completion, all the PRs used in this manual run will be updated to the status of the run (green on success, red otherwise).
Note: While trying to run this workflow manually, if you do not see an input field to one of the repos you want to use, just look in a different repo's actions. This is because GitHub only allows 10 input params, but all params should be covered in different repos.
If you need to update system tests according to a PR in some other repos, the protocol is:
- Make a feature branch on system tests repo with the updated tests
- Run system tests from any one of those repos manually against this feature branch, passing should make those PRs green.
- Merge all the other PRs in different repos.
- This feature branch will now pass against default config, and should be ready to merge.
The System Tests Pipeline can also run tests against an existing 0Chain network
[Click to show screenshot]
In this mode, supply the network URL. Docker image input fields will be ignored
Set the network URL field to the 0Chain network you wish to test, without the URL scheme or subdomain. eg. beta.0chain.net
The CI pipeline will generate an HTML report after test execution. In this report you can view logs from any test and see failures at a glance.
Requires BASH shell (UNIX, macOS, WSL) and go
Build or download the zbox and zwallet CLIs, ensuring they are compatible with the network you wish to test.
Modify the block_worker field in ./tests/cli_tests/config/zbox_config.yaml to point to the network.
To run the entire test suite (minus tests for known broken features) run:
cp $ZBOX_LOCATION ./tests/cli_tests/ # Copy zbox CLI to test folder
cp $ZWALLET_LOCATION ./tests/cli_tests/ # Copy zwallet CLI to test folder
cp $MC_LOCATION ./tests/cli_tests/ #Copy MC CLI to test folder
cp $WARP_LOCATION ./tests/cli_tests/ #Copy WARP CLI to test folder
cp $MINIO_LOCATION ./tests/cli_tests/ #Copy MINIO CLI to test folder
cd ./tests/cli_tests/
go test -run "^Test[^___]*$" ./... -vDebug logging can be achieved by running
DEBUG=true go test -run "^Test[^___]*$" ./... -vInclude tests for broken features as part of your test run by running
go test ./... -vPS: Test suite execution will be slower when running locally vs the system tests pipeline. Output will also be less clear vs the system tests pipeline. Therefore, we recommend using an IDE such as GoLand to run/debug individual tests locally
For developing new system tests for code still in developer branches, tests can be run against a locally running chain.
Typically, for a 0chain change you will have a PR for several modules that need to work
together. For example, 0chain, blobber, GoSDK, zboxcli and zwalletcli.
The first step requires setting up a running chain using the GitHub branches from the PRs. Use the instructions for building a local chain 0chain, add a few blobbers. Make sure you stake the blobbers.
For zboxcli and zwalletcli changes you need to first build the executable and copy into local
system test directory. For example:
cd zboxcli
make install
cp ./zbox ../system_test/tests/cli_tests/zbox
cd ../zwalletcli
make zwallet
cp ./zwallet ../system_test/tests/cli_tests/zwalletMake sure you have the correct system test branch. Now you need to edit system_test/tests/cli_tests/config/zbox_config.yaml
Edit the line block_worker: https://dev.0chain.net/dns to the appropriate setting for you, something like
block_worker: http://192.168.1.100:9091Now open the system_test project in GoLand,
you should now be able to run any of the cli_tests in debug.
You can run tests against a remote chain if you have already deployed elsewhere eg. dev.0chain.net
The test suite/pipeline should pass when ran against a healthy network. If some tests fail, it is likely that a code issue has been introduced. Try running the same tests against another network to rule out environmental issues. If the failure persists, and you believe this to be a false positive, contact the system tests team.
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.