Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
966eb57
ci(avalanchego-build)
Elvis339 Sep 25, 2025
ec8fe0e
ci: temp. run custom action on github runners
Elvis339 Sep 25, 2025
36f8420
ci: temp. run custom action on github runners with ubuntu-latest tag
Elvis339 Sep 25, 2025
8a517e5
ci(gh_runners): temp. checkout
Elvis339 Sep 25, 2025
04a9aa8
ci: remove checkout
Elvis339 Sep 25, 2025
1477677
ci: add checkout back
Elvis339 Sep 25, 2025
58784da
ci: remove defaults from coreth and libevm
Elvis339 Sep 25, 2025
bec87c7
debug
Elvis339 Sep 25, 2025
ebeafdc
ci: use $GITHUB_ACTION_PATH
Elvis339 Sep 25, 2025
17af2a7
ci: github runners
Elvis339 Sep 25, 2025
76b7009
ci: github runners use nix and use realpath
Elvis339 Sep 25, 2025
7aafbae
ci: github runners run help from bin
Elvis339 Sep 25, 2025
35e7eb6
ci: self-hosted runners
Elvis339 Sep 25, 2025
2ea5241
ci: self-hosted runners install build-essential
Elvis339 Sep 25, 2025
0e863be
ci: self-hosted runners run & build reexecution bench
Elvis339 Sep 25, 2025
b63ffe4
ci: self-hosted runners run & build reexecution bench
Elvis339 Sep 25, 2025
18e728c
ci(reexec): temp. disable go setup
Elvis339 Sep 25, 2025
3dfbc38
re-run re-exec
Elvis339 Sep 25, 2025
dfa8c81
ci: set permissions
Elvis339 Sep 25, 2025
06708b0
ci(avalanchego-build-action): use composable Firewood action
Elvis339 Sep 26, 2025
fb8a6cf
ci: reexecution bench with firewood custom action
Elvis339 Sep 26, 2025
da4c389
ci: add back ci jobs & remove tmp. jobs
Elvis339 Sep 26, 2025
6a09874
Merge branch 'master' into composable-ci-action
Elvis339 Sep 26, 2025
2c84ad9
ci: uncomment c-chain-reexecution-benchmark-* pull_request directive
Elvis339 Sep 26, 2025
f1a55c7
chore: address copilot PR review
Elvis339 Sep 26, 2025
4702f81
ci(c-chain-reexecution-benchmark)
Elvis339 Sep 26, 2025
7ac895c
ci(avalanchego-build-action): use `go get` instead of checking libevm…
Elvis339 Sep 26, 2025
26443d9
feat(build): extend build with Firewood
Elvis339 Sep 29, 2025
833f844
ci(avalanche-go-setup):
Elvis339 Sep 29, 2025
4257069
ci: test composition
Elvis339 Sep 29, 2025
24805ed
ci: test composition - switch AWS role
Elvis339 Sep 29, 2025
7503a0c
ci: test composition - switch AWS role
Elvis339 Sep 29, 2025
9f82ca8
ci: test composition - set permissions
Elvis339 Sep 29, 2025
db1bbc5
ci: test composition - set permissions
Elvis339 Sep 29, 2025
4df4b98
ci: test composition, set `config: firewood` to reexec bench
Elvis339 Sep 29, 2025
1674dfa
ci: set current state dir src to firewood 100
Elvis339 Sep 29, 2025
6de7ef2
ci: revert to default
Elvis339 Sep 29, 2025
6b212a9
Merge branch 'master' into composable-ci-action
Elvis339 Sep 29, 2025
07ada03
ci(avalanchego-setup-action): add `firewood-path` checkout path allow…
Elvis339 Oct 1, 2025
79f3d9b
Merge remote-tracking branch 'origin/composable-ci-action' into compo…
Elvis339 Oct 1, 2025
106ee40
ci(avalanchego-setup-action): handle checkout-path
Elvis339 Oct 1, 2025
57329f2
ci(avalanchego-setup-action): handle checkout-path
Elvis339 Oct 1, 2025
daff6b5
ci(avalanchego-setup-action): correct Firewood setup path
Elvis339 Oct 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/actions/avalanchego-setup-action/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# AvalancheGo Setup Action

## Overview
This action provides composable CI capabilities for setting up AvalancheGo with custom dependency versions.

### Why this exists?
Solves CI composability problems by enabling repositories to setup and build AvalancheGo with specific dependency versions without complex setup or cross-repository coordination.

### Why is it needed?
Dependencies need AvalancheGo as their integration context for realistic testing.
Without this action, setting up AvalancheGo with custom dependency versions requires build knowledge and manual `go mod` manipulation.
Teams either skip proper testing or dump tests in AvalancheGo (wrong ownership).
This action makes AvalancheGo composable - any repository can pull it in as integration context with one line.

## Security Model
- Repository Restriction: Only allows dependencies from `github.com/ava-labs/*` repositories
- No Custom Forks: Prevents supply chain attacks from malicious forks
- Commit Validation: Validates dependency versions reference ava-labs repositories only

## Inputs

| Input | Description | Required | Default |
|-------|-------------|----------|-----------------------|
| `checkout-path` | Directory path where AvalancheGo will be checked out | No | `'.'` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this suggested?

| `avalanchego` | AvalancheGo version (commit SHA, branch, tag) | No | `'${{ github.sha }}'` |
| `firewood` | Firewood version. Consumer should run Firewood shared workflow first | No | `''` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's the firewood dir?

| `coreth` | Coreth version (commit SHA, branch, tag) | No | `''` |
| `libevm` | LibEVM version (commit SHA, branch, tag) | No | `''` |

## Usage Examples

```yaml
- name: Setup AvalancheGo with Firewood # This will setup go.mod
uses: ./.github/actions/avalanchego-setup-action
with:
checkout-path: "build/avalanchego"
coreth: "my-feature-branch"
libevm: "experimental-branch"
firewood: "ffi/v0.0.12"
- name: Load test # This will compile and run the load test
uses: ./.github/actions/run-monitored-tmpnet-cmd
with:
run: ./scripts/run_task.sh test-load -- --load-timeout=30m --firewood
```
61 changes: 61 additions & 0 deletions .github/actions/avalanchego-setup-action/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: 'Setup AvalancheGo'
description: 'Setup AvalancheGo with custom dependencies (Firewood, Coreth, LibEVM)'

inputs:
avalanchego:
description: 'AvalancheGo version (commit SHA, branch name, or tag)'
required: false
default: ${{ github.sha }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reference only makes sense if the custom action will only be used in the avalanchego repo.

checkout-path:
description: 'Path where AvalancheGo will be checked out'
required: false
default: '.'
firewood:
description: 'Firewood version (commit SHA, branch, tag, or ffi/vX.Y.Z for pre-built)'
required: false
default: ''
firewood-path:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under what circumstances will this actually be used?

description: 'Directory path where Firewood will be cloned/built (preferably NVMe storage)'
required: false
default: ''
coreth:
description: 'Coreth version (commit SHA, branch name, or tag)'
required: false
default: ''
libevm:
description: 'LibEVM version (commit SHA, branch name, or tag)'
required: false
default: ''

runs:
using: 'composite'
steps:
- name: Checkout AvalancheGo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only non-task step that I would expect to see in an action with the remain steps being part of a task.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point. I can see the appeal of consolidating all go.mod updates in one place.

I'm thinking of keeping coreth/libevm updates explicit in the action makes the dependency changes visible and optional. They're simple go get commands, so putting them in setup_firewood.sh would add context/coupling without clear benefit - the script would need to handle optional arguments for dependencies it doesn't own.

That said, I'm not strongly opposed to consolidating them. Do you see a clear advantage to moving them into the script? Or is this more about having a consistent pattern for where dependency management happens?

Happy to restructure if there's a compelling reason I'm missing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern for the majority of our CI jobs is that they are trivial to reproduce locally. That means, apart from CI-specific setup like cloning, caching and installing non-nix deps, the meat of a given job is encapsulated in a task that can be invoked locally. CI jobs are then just the glue between Github Actions and our tasks. This ensures both reproducibility and simplifies development and maintenance of our jobs.

uses: actions/checkout@v4
with:
repository: 'ava-labs/avalanchego'
ref: ${{ inputs.avalanchego }}
path: ${{ inputs.checkout-path }}
- name: Update Coreth dependency
if: inputs.coreth != ''
shell: bash
working-directory: ${{ inputs.checkout-path }}
run: |
echo "Updating Coreth to version: ${{ inputs.coreth }}"
go get github.com/ava-labs/coreth@${{ inputs.coreth }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What guarantees are there that a given version of coreth is going to be compatible with given versions of avalanchego or firewood?

- name: Update LibEVM dependency
if: inputs.libevm != ''
shell: bash
working-directory: ${{ inputs.checkout-path }}
run: |
echo "Updating LibEVM to version: ${{ inputs.libevm }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What guarantees are there that a given version of libevm is going to be compatible with given versions of avalanchego and coreth?

go get github.com/ava-labs/libevm@${{ inputs.libevm }}
- name: Setup Firewood FFI
if: inputs.firewood != ''
shell: bash
run: |
if [ -n "${{ inputs.firewood-path }}" ]; then
"${{ inputs.checkout-path }}/scripts/setup_firewood.sh" "${{ inputs.firewood }}" "${{ inputs.firewood-path }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counter to my insistence on a task being the preferred entrypoint, the only way this custom action is going to be able to be used in other repos isif they have this same script in the same location, or the script is in the action dir and referred to with a relative path. Which doesn't preclude the use of a task, said task would just need to point to the updated script path.

else
"${{ inputs.checkout-path }}/scripts/setup_firewood.sh" "${{ inputs.firewood }}"
fi
6 changes: 6 additions & 0 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ tasks:
desc: Builds avalanchego
cmd: ./scripts/build.sh

build-with-firewood:
desc: Builds avalanchego with Firewood FFI (specify version with FIREWOOD_VERSION)
vars:
FIREWOOD_VERSION: '{{.FIREWOOD_VERSION | default "ffi/v0.0.12"}}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make it clear that what is being accepted here is any of branch|tag|SHA?

cmd: ./scripts/build.sh -f {{.FIREWOOD_VERSION}}

build-antithesis-images-avalanchego:
desc: Builds docker images for antithesis for the avalanchego test setup
env:
Expand Down
22 changes: 18 additions & 4 deletions scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,29 @@ print_usage() {

Options:

-r Build with race detector
-r Build with race detector
-f VERSION Build with Firewood FFI
VERSION format: ffi/vX.Y.Z for pre-built, commit/branch for source
"
}

race=''
while getopts 'r' flag; do
firewood_version=''

while getopts 'rf:' flag; do
case "${flag}" in
r)
echo "Building with race detection enabled"
race='-race'
;;
*) print_usage
exit 1 ;;
f)
firewood_version="${OPTARG}"
echo "Building with Firewood version: ${firewood_version}"
;;
*)
print_usage
exit 1
;;
esac
done

Expand All @@ -31,6 +41,10 @@ source "${REPO_ROOT}"/scripts/constants.sh
# Determine the git commit hash to use for the build
source "${REPO_ROOT}"/scripts/git_commit.sh

if [ -n "${firewood_version}" ]; then
"${REPO_ROOT}/scripts/setup_firewood.sh" "${firewood_version}" "${REPO_ROOT}"
fi

echo "Building AvalancheGo with [$(go version)]..."
go build ${race} -o "${avalanchego_path}" \
-ldflags "-X github.com/ava-labs/avalanchego/version.GitCommit=$git_commit $static_ld_flags" \
Expand Down
77 changes: 77 additions & 0 deletions scripts/setup_firewood.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#!/usr/bin/env bash

# Setup Firewood FFI
#
# Clones Firewood repository, builds/fetches the FFI, and updates go.mod
#
# Usage:
# setup_firewood.sh <version> [workspace]
#
# Arguments:
# version Firewood version (ffi/vX.Y.Z for pre-built, commit/branch for source)
# workspace Optional workspace path for Firewood build (default: ${AVALANCHE_PATH}/firewood-workspace)
#
# Output:
# Prints FFI path to stdout on success

set -euo pipefail

if [ $# -lt 1 ]; then
echo "Usage: $0 <version> [workspace]" >&2
exit 1
fi

FIREWOOD_VERSION="$1"
AVALANCHE_PATH=$( cd "$( dirname "${BASH_SOURCE[0]}" )"; cd .. && pwd )
FIREWOOD_CLONE_DIR="${AVALANCHE_PATH}/firewood"

# Use provided workspace or default to avalanchego/firewood-workspace
if [ $# -ge 2 ] && [ -n "$2" ]; then
WORKSPACE_PATH="$2"
else
WORKSPACE_PATH="${AVALANCHE_PATH}/firewood-workspace"
fi

if [ -d "${FIREWOOD_CLONE_DIR}" ]; then
echo "Removing existing Firewood directory..." >&2
rm -rf "${FIREWOOD_CLONE_DIR}"
fi

echo "Setting up Firewood FFI version: ${FIREWOOD_VERSION}" >&2
echo "Using workspace: ${WORKSPACE_PATH}" >&2

git clone https://github.com/ava-labs/firewood "${FIREWOOD_CLONE_DIR}" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it desirable to clone just to get the file rather than using a raw url? And then the build-firewood.sh script will go off and do a full clone anyway.

I'm not sure why this would be at all preferable to just building with nix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. We can leverage building with nix once it's merged.

--quiet --depth 1 --branch composable-ci-action

SETUP_FIREWOOD_SCRIPT="${FIREWOOD_CLONE_DIR}/benchmark/setup-scripts/build-firewood.sh"

if [ ! -f "${SETUP_FIREWOOD_SCRIPT}" ]; then
echo "Error: Setup Firewood script not found at ${SETUP_FIREWOOD_SCRIPT}" >&2
exit 1
fi

# Build or fetch Firewood FFI with custom workspace
# Capture only the last line which is the FFI path
FFI_PATH=$("${SETUP_FIREWOOD_SCRIPT}" "${FIREWOOD_VERSION}" --workspace "${WORKSPACE_PATH}" | tail -n 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not appear that the target script accepts a version. afaik only way to build a specific version would be to clone at said version. Is there an updated script that hasn't yet merged to firewood main?


if [ -z "${FFI_PATH}" ]; then
echo "Error: Failed to build/fetch Firewood FFI" >&2
exit 1
fi

cd "${AVALANCHE_PATH}"

# Verify go.mod exists
if [ ! -f "go.mod" ]; then
echo "Error: go.mod not found in ${AVALANCHE_PATH}" >&2
exit 1
fi

echo "Updating go.mod with FFI path: ${FFI_PATH}" >&2
go mod edit -replace github.com/ava-labs/firewood-go-ethhash/ffi="${FFI_PATH}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not ideal that after I execute this script, my working tree is now dirty and I have to make sure I don't accidentally commit the change.


go mod tidy
go mod download

# Output FFI path to stdout for consumption by other scripts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you intending that the script call itself? I don't see any other case of tailing the output to get the path. Maybe just assume $PWD/firewood/ffi?

echo "${FFI_PATH}"