-
Notifications
You must be signed in to change notification settings - Fork 808
ci: composable avalanchego action #4341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
966eb57
ec8fe0e
36f8420
8a517e5
04a9aa8
1477677
58784da
bec87c7
ebeafdc
17af2a7
76b7009
7aafbae
35e7eb6
2ea5241
0e863be
b63ffe4
18e728c
3dfbc38
dfa8c81
06708b0
fb8a6cf
da4c389
6a09874
2c84ad9
f1a55c7
4702f81
7ac895c
26443d9
833f844
4257069
24805ed
7503a0c
9f82ca8
db1bbc5
4df4b98
1674dfa
6de7ef2
6b212a9
07ada03
79f3d9b
106ee40
57329f2
daff6b5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# AvalancheGo Setup Action | ||
|
||
## Overview | ||
This action provides composable CI capabilities for setting up AvalancheGo with custom dependency versions. | ||
|
||
### Why this exists? | ||
Solves CI composability problems by enabling repositories to setup and build AvalancheGo with specific dependency versions without complex setup or cross-repository coordination. | ||
|
||
### Why is it needed? | ||
Dependencies need AvalancheGo as their integration context for realistic testing. | ||
Without this action, setting up AvalancheGo with custom dependency versions requires build knowledge and manual `go mod` manipulation. | ||
Teams either skip proper testing or dump tests in AvalancheGo (wrong ownership). | ||
This action makes AvalancheGo composable - any repository can pull it in as integration context with one line. | ||
|
||
## Security Model | ||
- Repository Restriction: Only allows dependencies from `github.com/ava-labs/*` repositories | ||
- No Custom Forks: Prevents supply chain attacks from malicious forks | ||
- Commit Validation: Validates dependency versions reference ava-labs repositories only | ||
|
||
## Inputs | ||
|
||
| Input | Description | Required | Default | | ||
|-------|-------------|----------|-----------------------| | ||
| `checkout-path` | Directory path where AvalancheGo will be checked out | No | `'.'` | | ||
| `avalanchego` | AvalancheGo version (commit SHA, branch, tag) | No | `'${{ github.sha }}'` | | ||
| `firewood` | Firewood version. Consumer should run Firewood shared workflow first | No | `''` | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where's the firewood dir? |
||
| `coreth` | Coreth version (commit SHA, branch, tag) | No | `''` | | ||
| `libevm` | LibEVM version (commit SHA, branch, tag) | No | `''` | | ||
|
||
## Usage Examples | ||
|
||
```yaml | ||
- name: Setup AvalancheGo with Firewood # This will setup go.mod | ||
uses: ./.github/actions/avalanchego-setup-action | ||
with: | ||
checkout-path: "build/avalanchego" | ||
coreth: "my-feature-branch" | ||
libevm: "experimental-branch" | ||
firewood: "ffi/v0.0.12" | ||
- name: Load test # This will compile and run the load test | ||
uses: ./.github/actions/run-monitored-tmpnet-cmd | ||
with: | ||
run: ./scripts/run_task.sh test-load -- --load-timeout=30m --firewood | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
name: 'Setup AvalancheGo' | ||
description: 'Setup AvalancheGo with custom dependencies (Firewood, Coreth, LibEVM)' | ||
|
||
inputs: | ||
avalanchego: | ||
description: 'AvalancheGo version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: ${{ github.sha }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This reference only makes sense if the custom action will only be used in the avalanchego repo. |
||
checkout-path: | ||
description: 'Path where AvalancheGo will be checked out' | ||
required: false | ||
default: '.' | ||
firewood: | ||
description: 'Firewood version (commit SHA, branch, tag, or ffi/vX.Y.Z for pre-built)' | ||
required: false | ||
default: '' | ||
firewood-path: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Under what circumstances will this actually be used? |
||
description: 'Directory path where Firewood will be cloned/built (preferably NVMe storage)' | ||
required: false | ||
default: '' | ||
coreth: | ||
description: 'Coreth version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: '' | ||
libevm: | ||
description: 'LibEVM version (commit SHA, branch name, or tag)' | ||
required: false | ||
default: '' | ||
|
||
runs: | ||
using: 'composite' | ||
steps: | ||
- name: Checkout AvalancheGo | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the only non-task step that I would expect to see in an action with the remain steps being part of a task. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a fair point. I can see the appeal of consolidating all go.mod updates in one place. I'm thinking of keeping coreth/libevm updates explicit in the action makes the dependency changes visible and optional. They're simple That said, I'm not strongly opposed to consolidating them. Do you see a clear advantage to moving them into the script? Or is this more about having a consistent pattern for where dependency management happens? Happy to restructure if there's a compelling reason I'm missing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The pattern for the majority of our CI jobs is that they are trivial to reproduce locally. That means, apart from CI-specific setup like cloning, caching and installing non-nix deps, the meat of a given job is encapsulated in a task that can be invoked locally. CI jobs are then just the glue between Github Actions and our tasks. This ensures both reproducibility and simplifies development and maintenance of our jobs. |
||
uses: actions/checkout@v4 | ||
with: | ||
repository: 'ava-labs/avalanchego' | ||
ref: ${{ inputs.avalanchego }} | ||
path: ${{ inputs.checkout-path }} | ||
- name: Update Coreth dependency | ||
if: inputs.coreth != '' | ||
shell: bash | ||
working-directory: ${{ inputs.checkout-path }} | ||
run: | | ||
echo "Updating Coreth to version: ${{ inputs.coreth }}" | ||
go get github.com/ava-labs/coreth@${{ inputs.coreth }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What guarantees are there that a given version of coreth is going to be compatible with given versions of avalanchego or firewood? |
||
- name: Update LibEVM dependency | ||
if: inputs.libevm != '' | ||
shell: bash | ||
working-directory: ${{ inputs.checkout-path }} | ||
run: | | ||
echo "Updating LibEVM to version: ${{ inputs.libevm }}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What guarantees are there that a given version of libevm is going to be compatible with given versions of avalanchego and coreth? |
||
go get github.com/ava-labs/libevm@${{ inputs.libevm }} | ||
- name: Setup Firewood FFI | ||
if: inputs.firewood != '' | ||
shell: bash | ||
run: | | ||
if [ -n "${{ inputs.firewood-path }}" ]; then | ||
"${{ inputs.checkout-path }}/scripts/setup_firewood.sh" "${{ inputs.firewood }}" "${{ inputs.firewood-path }}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Counter to my insistence on a task being the preferred entrypoint, the only way this custom action is going to be able to be used in other repos isif they have this same script in the same location, or the script is in the action dir and referred to with a relative path. Which doesn't preclude the use of a task, said task would just need to point to the updated script path. |
||
else | ||
"${{ inputs.checkout-path }}/scripts/setup_firewood.sh" "${{ inputs.firewood }}" | ||
fi |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,6 +11,12 @@ tasks: | |
desc: Builds avalanchego | ||
cmd: ./scripts/build.sh | ||
|
||
build-with-firewood: | ||
desc: Builds avalanchego with Firewood FFI (specify version with FIREWOOD_VERSION) | ||
vars: | ||
FIREWOOD_VERSION: '{{.FIREWOOD_VERSION | default "ffi/v0.0.12"}}' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe make it clear that what is being accepted here is any of |
||
cmd: ./scripts/build.sh -f {{.FIREWOOD_VERSION}} | ||
|
||
build-antithesis-images-avalanchego: | ||
desc: Builds docker images for antithesis for the avalanchego test setup | ||
env: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Setup Firewood FFI | ||
# | ||
# Clones Firewood repository, builds/fetches the FFI, and updates go.mod | ||
# | ||
# Usage: | ||
# setup_firewood.sh <version> [workspace] | ||
# | ||
# Arguments: | ||
# version Firewood version (ffi/vX.Y.Z for pre-built, commit/branch for source) | ||
# workspace Optional workspace path for Firewood build (default: ${AVALANCHE_PATH}/firewood-workspace) | ||
# | ||
# Output: | ||
# Prints FFI path to stdout on success | ||
|
||
set -euo pipefail | ||
|
||
if [ $# -lt 1 ]; then | ||
echo "Usage: $0 <version> [workspace]" >&2 | ||
exit 1 | ||
fi | ||
|
||
FIREWOOD_VERSION="$1" | ||
AVALANCHE_PATH=$( cd "$( dirname "${BASH_SOURCE[0]}" )"; cd .. && pwd ) | ||
FIREWOOD_CLONE_DIR="${AVALANCHE_PATH}/firewood" | ||
|
||
# Use provided workspace or default to avalanchego/firewood-workspace | ||
if [ $# -ge 2 ] && [ -n "$2" ]; then | ||
WORKSPACE_PATH="$2" | ||
else | ||
WORKSPACE_PATH="${AVALANCHE_PATH}/firewood-workspace" | ||
fi | ||
|
||
if [ -d "${FIREWOOD_CLONE_DIR}" ]; then | ||
echo "Removing existing Firewood directory..." >&2 | ||
rm -rf "${FIREWOOD_CLONE_DIR}" | ||
fi | ||
|
||
echo "Setting up Firewood FFI version: ${FIREWOOD_VERSION}" >&2 | ||
echo "Using workspace: ${WORKSPACE_PATH}" >&2 | ||
|
||
git clone https://github.com/ava-labs/firewood "${FIREWOOD_CLONE_DIR}" \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is it desirable to clone just to get the file rather than using a raw url? And then the I'm not sure why this would be at all preferable to just building with nix. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the suggestion. We can leverage building with nix once it's merged. |
||
--quiet --depth 1 --branch composable-ci-action | ||
|
||
SETUP_FIREWOOD_SCRIPT="${FIREWOOD_CLONE_DIR}/benchmark/setup-scripts/build-firewood.sh" | ||
|
||
if [ ! -f "${SETUP_FIREWOOD_SCRIPT}" ]; then | ||
echo "Error: Setup Firewood script not found at ${SETUP_FIREWOOD_SCRIPT}" >&2 | ||
exit 1 | ||
fi | ||
|
||
# Build or fetch Firewood FFI with custom workspace | ||
# Capture only the last line which is the FFI path | ||
FFI_PATH=$("${SETUP_FIREWOOD_SCRIPT}" "${FIREWOOD_VERSION}" --workspace "${WORKSPACE_PATH}" | tail -n 1) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It does not appear that the target script accepts a version. afaik only way to build a specific version would be to clone at said version. Is there an updated script that hasn't yet merged to firewood main? |
||
|
||
if [ -z "${FFI_PATH}" ]; then | ||
echo "Error: Failed to build/fetch Firewood FFI" >&2 | ||
exit 1 | ||
fi | ||
|
||
cd "${AVALANCHE_PATH}" | ||
|
||
# Verify go.mod exists | ||
if [ ! -f "go.mod" ]; then | ||
echo "Error: go.mod not found in ${AVALANCHE_PATH}" >&2 | ||
exit 1 | ||
fi | ||
|
||
echo "Updating go.mod with FFI path: ${FFI_PATH}" >&2 | ||
go mod edit -replace github.com/ava-labs/firewood-go-ethhash/ffi="${FFI_PATH}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is not ideal that after I execute this script, my working tree is now dirty and I have to make sure I don't accidentally commit the change. |
||
|
||
go mod tidy | ||
go mod download | ||
|
||
# Output FFI path to stdout for consumption by other scripts | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are you intending that the script call itself? I don't see any other case of tailing the output to get the path. Maybe just assume |
||
echo "${FFI_PATH}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this suggested?