Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add bench-bot infra docs #58

Merged
merged 2 commits into from
Jul 1, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ many repositories) for our CI pipelines.
- [Usage](#check_dependent_project-usage)
- [Explanation](#check_dependent_project-explanation)
- [Implementation](#check_dependent_project-implementation)
- [bench-bot](#bench_bot)
- [Infrastructure](#bench_bot_infra)

# check_dependent_project <a name="check_dependent_project"></a>

Expand Down Expand Up @@ -144,3 +146,73 @@ typechecking would fail.

The check is currently limited to `cargo check` but optimally it should
[entail the whole CI pipeline of `$dependent_repo`](https://github.com/paritytech/ci_cd/issues/234).

# bench-bot <a name="bench_bot"></a>
- Bot that runs benchmark commands for users .
- Uses gitlab-runner to run the jobs.
- Users posts a comment on a PR in substrate repo with the command that needs to run and the bot runs it.

E. G. `/cmd queue -c bench-bot $ pallet dev pallet_balances`
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved

## Infrastrcture <a name="#bench_bot_infra"></a>

This bot runs on [bm3 machine](https://gitlab.parity.io/parity/devops/-/blob/master/inventory.yaml#L485),
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
currently maintained by CI/CD team.

### Configuration requirements:
The machine needs to configred as a production gitlab-runer.
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved

Config requirements:
* gitlab_runner_executor: shell
* gitlab_runner_additional_tags: bench-bot
* os: ubuntu20
* provider: ovh
* HOME for `gitlab-runner` needs to be in `/home` because this is monunted on a disk witk more space then the root partition
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
and there will be a lot of rust code compiled
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
* install rustup for gitlab-runner user
* set default rustup
* install toolchain nightly
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
* add wasm target
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
* install cmake

All those requirements are saved in an ansible role called `rustup`
TODO: add link to rustup role once the PR is merged

### Ansible
Because there is only one host running code for this bot, one needs to create
a test VM and add it to inventory everywhere `bm3` is mentiond.
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved

```
test-bench-bot:
ansible_host: 34.141.84.244
os: ubuntu20
provider: ovh
gitlab_runner_executor: shell
gitlab_runner_additional_tags: bench-bot
### required for rustup ransible role
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
# Install rustup, and rust toolchain as defined value.
rustup_user: gitlab-runner
# Adds {HOME/.cargo/bin} to user's PATH
rustup_configure_shell: true
# do not install any cargo crates
rustup_cargo_crates: []
```

**When creating the VM helps a lot if you include your ssh public key**
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
```
/home/radu/.ssh/id_rsa_yubikey.pub
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved
```

Test the ansible role from `parity/devops`
```
ansible-playbook ansible/gitlab-runner.yaml -l test-bench-bot -u radupopa2010 -t gitlab-runner-benchmarks-weights-provision
```

### Test

In order not to disrupt the service, JP created a test command that uses gitlab tag `test-bench-bot`
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved

1. Fisrt create a dummy PR in substrate repo like [this one](https://github.com/paritytech/substrate/pull/11566)
2. Add a comment `/cmd queue -c test-bench-bot $ pallet dev pallet_balances`
3. Wait a bit and check for a commnet from the bot with the job that started
radupopa2010 marked this conversation as resolved.
Show resolved Hide resolved