Skip to content

Commit

Permalink
finalize pipeline, trigger
Browse files Browse the repository at this point in the history
  • Loading branch information
skudasov committed Nov 12, 2024
1 parent 934dcc8 commit 65ca88e
Show file tree
Hide file tree
Showing 11 changed files with 28 additions and 111 deletions.
12 changes: 4 additions & 8 deletions .github/workflows/framework-golden-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,16 @@ jobs:
config: smoke.toml
count: 1
timeout: 10m
- name: TestLoad
config: load.toml
- name: TestPerformanceBaseline
config: performance_baseline.toml
count: 1
timeout: 10m
- name: TestChaos
config: chaos.toml
count: 1
timeout: 10m
- name: TestUpgradeAll
config: upgrade_all.toml
count: 1
timeout: 10m
- name: TestUpgradeSome
config: upgrade_some.toml
- name: TestUpgrade
config: upgrade.toml
count: 1
timeout: 10m
steps:
Expand Down
2 changes: 1 addition & 1 deletion book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
- [Chainlink]()
- [RPC]()
- [Loki]()
- [Testing](framework/testing.md)
- [Testing Maturity Model](framework/testing.md)
- [Smoke]()
- [Performance]()
- [Chaos]()
Expand Down
19 changes: 8 additions & 11 deletions book/src/framework/testing.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,21 @@
# "Golden" Templates
# Testing Maturity Model

[Here](https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/job/32703095118?pr=1311) are our "golden" templates for end-to-end tests, covering every test type:

- `Smoke`
- `Performance`
- `PerformanceBaseline`
- `Chaos`
- `Upgrade`

These tests act as a maturity model and are implemented across all our products.

Run them locally:
Refer to this README to understand the rationale behind our testing approach and to explore the stages of maturity in end-to-end testing.

## Developing
Run the tests locally
```
CTF_CONFIGS=smoke.toml go test -v -run TestSmoke
CTF_CONFIGS=load.toml go test -v -run TestLoad
CTF_CONFIGS=performance_baseline.toml go test -v -run TestPerformanceBaseline
CTF_CONFIGS=chaos.toml go test -v -run TestChaos
CTF_CONFIGS=upgrade_some.toml go test -v -run TestUpgradeSome
CTF_CONFIGS=upgrade.toml go test -v -run TestUpgrade
```

Use this [workflow](https://github.com/smartcontractkit/chainlink-testing-framework/actions/runs/11739154666/workflow?pr=1311) as a starting point for developing a new end-to-end integration test.

Set the count to 5-10 during development, and once stable, set the timeout and proceed to merge.

If you need to structure a lot of different tests (not only end-to-end) follow [this](https://github.com/smartcontractkit/.github/tree/main/.github/workflows) guide.
8 changes: 2 additions & 6 deletions framework/examples/myproject/.envrc
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
export LOKI_TENANT_ID=promtail
export LOKI_URL=http://localhost:3030/loki/api/v1/push
export TESTCONTAINERS_RYUK_DISABLED=true
export PRIVATE_KEY="..."
# load test
#export LOKI_TENANT_ID=promtail
#export LOKI_URL=http://localhost:3030/loki/api/v1/push
#export RESTY_DEBUG=true
export CTF_LOG_LEVEL=info
11 changes: 10 additions & 1 deletion framework/examples/myproject/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,10 @@
# End-to-End Testing Project Maturity Model
# End-to-End Testing Project Maturity Model

## Developing
Run the tests locally
```
CTF_CONFIGS=smoke.toml go test -v -run TestSmoke
CTF_CONFIGS=performance_baseline.toml go test -v -run TestPerformanceBaseline
CTF_CONFIGS=chaos.toml go test -v -run TestChaos
CTF_CONFIGS=upgrade.toml go test -v -run TestUpgrade
```
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ type CfgLoad struct {
NodeSet *ns.Input `toml:"nodeset" validate:"required"`
}

func TestLoad(t *testing.T) {
func TestPerformanceBaseline(t *testing.T) {
in, err := framework.Load[CfgLoad](t)
require.NoError(t, err)

Expand Down
File renamed without changes.
23 changes: 0 additions & 23 deletions framework/examples/myproject/upgrade_all.toml

This file was deleted.

58 changes: 0 additions & 58 deletions framework/examples/myproject/upgrade_all_test.go

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ type CfgReload struct {
NodeSet *ns.Input `toml:"nodeset" validate:"required"`
}

func TestUpgradeSome(t *testing.T) {
func TestUpgrade(t *testing.T) {
in, err := framework.Load[CfgReload](t)
require.NoError(t, err)

Expand Down Expand Up @@ -66,7 +66,7 @@ func TestUpgradeSome(t *testing.T) {
level = 'info'
`

out, err = ns.UpgradeNodeSet(in.NodeSet, bc, dp.BaseURLDocker, 1*time.Second)
out, err = ns.UpgradeNodeSet(in.NodeSet, bc, dp.BaseURLDocker, 3*time.Second)
require.NoError(t, err)

jobs, _, err := c[0].ReadJobs()
Expand Down

0 comments on commit 65ca88e

Please sign in to comment.