Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(ci): integrate arm64 runners + zig build #2130

Merged
merged 9 commits into from
Jan 22, 2025
Merged

refactor(ci): integrate arm64 runners + zig build #2130

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fe10874
update(deps): bump ncurses to 6.5
therealbobo Dec 10, 2024
11e2640
feat(ci): use zig for compilation
therealbobo Dec 10, 2024
3a28ef3
cleanup: remove old docker builder files
therealbobo Dec 14, 2024
0ae2d5f
chore: remove old files
therealbobo Jan 16, 2025
07982f6
new: add arm workers
therealbobo Jan 16, 2025
ab6bfa9
fix(ci): bump updload-artifact to v4
therealbobo Jan 16, 2025
4efe8a1
fix(ci): don't use builder image
therealbobo Jan 21, 2025
bee9596
cleanup: remove deprecated stuff
therealbobo Jan 21, 2025
2cd4f4f
refactor(ci): integrate arm64 runner + clenaup
therealbobo Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
302 changes: 82 additions & 220 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,238 +8,104 @@


jobs:
builder:
build-sysdig-linux:
runs-on: ubuntu-24.04${{ matrix.platform == 'arm64' && '-arm' || '' }}
container:
image: ubuntu:22.04
strategy:
matrix:
platform:
- amd64
- arm64
env:
REGISTRY: ghcr.io
BUILDER_IMAGE_BASE: ghcr.io/draios/sysdig-builder-pr
SKEL_BUILDER_IMAGE_BASE: ghcr.io/draios/sysdig-skel-builder-pr
BUILDER_DEV: ghcr.io/draios/sysdig-builder:dev
SKEL_BUILDER_DEV: ghcr.io/draios/sysdig-skel-builder:dev
ZIG_VERSION: 0.14.0-dev.2851+b074fb7dd

runs-on: ubuntu-latest
steps:
- name: Checkout Sysdig
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Check if builder is modified
id: builder-files
uses: tj-actions/changed-files@v34
with:
files: |
docker/builder/**

- name: Set up QEMU
if: steps.builder-files.outputs.any_changed == 'true'
uses: docker/setup-qemu-action@v3
with:
platforms: 'amd64,arm64'

- name: Set up Docker Buildx
if: steps.builder-files.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@v2

- name: Login to Github Packages
if: steps.builder-files.outputs.any_changed == 'true'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Get new skeleton builder image tag
id: get-new-skeleton-builder
if: steps.builder-files.outputs.any_changed == 'true'
- name: Install deps
run: |

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2046:warning:28:47: Quote this to prevent word splitting [shellcheck]

Raw Output:
w:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2046:warning:28:47: Quote this to prevent word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:28:59: Double quote to prevent globbing and word splitting [shellcheck]

Raw Output:
i:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2086:info:28:59: Double quote to prevent globbing and word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2046:warning:29:20: Quote this to prevent word splitting [shellcheck]

Raw Output:
w:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2046:warning:29:20: Quote this to prevent word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:29:32: Double quote to prevent globbing and word splitting [shellcheck]

Raw Output:
i:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2086:info:29:32: Double quote to prevent globbing and word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2046:warning:30:17: Quote this to prevent word splitting [shellcheck]

Raw Output:
w:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2046:warning:30:17: Quote this to prevent word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:30:29: Double quote to prevent globbing and word splitting [shellcheck]

Raw Output:
i:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2086:info:30:29: Double quote to prevent globbing and word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2046:warning:31:14: Quote this to prevent word splitting [shellcheck]

Raw Output:
w:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2046:warning:31:14: Quote this to prevent word splitting [shellcheck]

Check warning on line 30 in .github/workflows/ci.yaml

View workflow job for this annotation

GitHub Actions / actionlint 

[actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:31:26: Double quote to prevent globbing and word splitting [shellcheck]

Raw Output:
i:.github/workflows/ci.yaml:30:9: shellcheck reported issue in this script: SC2086:info:31:26: Double quote to prevent globbing and word splitting [shellcheck]
echo "skeleton_builder_image=${{ (github.event.pull_request.number != '') && format('{0}:{1}', env.SKEL_BUILDER_IMAGE_BASE, github.event.pull_request.number) || env.SKEL_BUILDER_DEV }}" >> $GITHUB_OUTPUT

- name: Get new builder image tag
id: get-new-builder
if: steps.builder-files.outputs.any_changed == 'true'
cp -v scripts/zig-cc /usr/bin/
cp -v scripts/zig-c++ /usr/bin/
apt update && \
apt install -y --no-install-recommends \
autoconf \
automake \
build-essential \
ca-certificates \
clang \
cmake \
curl \
git \
libelf-dev \
libtool \
llvm \
ninja-build \
pkg-config \
rpm \
wget \
xz-utils && \
git clone https://github.com/libbpf/bpftool.git --branch v7.3.0 --single-branch && \
cd bpftool && \
git submodule update --init && \
cd src && \
make install && \
cd ../.. && \
rm -fr bpftool && \
curl -LO https://ziglang.org/builds/zig-linux-$(uname -m)-${ZIG_VERSION}.tar.xz && \
tar -xaf zig-linux-$(uname -m)-${ZIG_VERSION}.tar.xz && \
rm -v zig-linux-$(uname -m)-${ZIG_VERSION}.tar.xz && \
cd zig-linux-$(uname -m)-${ZIG_VERSION} && \
cp -v zig /usr/bin && \
find lib -exec cp --parents {} /usr/ \; && \
cd .. && \
rm -fr zig*

- name: Build Sysdig
env:
CC: zig-cc
CXX: zig-c++
AR: zig ar
RANLIB: zig ranlib
run: |
echo "builder_image=${{ (github.event.pull_request.number != '') && format('{0}:{1}', env.BUILDER_IMAGE_BASE, github.event.pull_request.number) || env.BUILDER_DEV }}" >> $GITHUB_OUTPUT

- name: Build new skeleton builder
id: skeleton-builder
if: steps.builder-files.outputs.any_changed == 'true'
uses: docker/build-push-action@v6
with:
context: docker/builder
file: docker/builder/modern_bpf_probe.Dockerfile
platforms: linux/amd64,linux/arm64
tags: ${{ steps.get-new-skeleton-builder.outputs.skeleton_builder_image }}
push: true

- name: Build new builder
id: build-builder
if: steps.builder-files.outputs.any_changed == 'true'
uses: docker/build-push-action@v6
with:
context: docker/builder
platforms: linux/amd64,linux/arm64
tags: ${{ steps.get-new-builder.outputs.builder_image }}
push: true

outputs:
builder_image: ${{ (steps.builder-files.outputs.any_changed == 'true') && steps.get-new-builder.outputs.builder_image || env.BUILDER_DEV }}
skeleton_builder_image: ${{ (steps.builder-files.outputs.any_changed == 'true') && steps.get-new-skeleton-builder.outputs.skeleton_builder_image || env.SKEL_BUILDER_DEV }}

build-skeleton-sysdig-linux-amd64:
needs: builder
runs-on: ubuntu-latest
container:
image: ${{ needs.builder.outputs.skeleton_builder_image }}
steps:
- name: Checkout Sysdig
uses: actions/checkout@v4
with:
path: sysdig
- name: Link paths
cmake \
-DUSE_BUNDLED_DEPS=ON \
-DBUILD_BPF=OFF \
-DBUILD_DRIVER=OFF \
-DCMAKE_BUILD_TYPE=Release \
-S . \
-B build \
-G Ninja
cmake --build build --target package --config Release

- name: Set artifact name
id: artifact_name
run: |
mkdir -p /source
ln -s "$GITHUB_WORKSPACE/sysdig" /source/sysdig
- name: Prepare build skeleton
run: build cmake-skeleton
- name: Build skeleton
run: build make-skeleton
- name: Cache build skeleton
uses: actions/cache/save@v3
if: always()
id: cache
with:
path: /build-skeleton
key: build-skeleton-${{ github.run_id }}
echo "name=sysdig-dev-linux-$(echo ${{ matrix.platform }} | sed -e 's|/|-|g')" >> "$GITHUB_OUTPUT"

build-sysdig-linux-amd64:
needs: [builder,build-skeleton-sysdig-linux-amd64]
runs-on: ubuntu-latest
container:
image: ${{ needs.builder.outputs.builder_image }}
volumes:
- '/node20217:/node20217:rw,rshared'
- '/node20217:/__e/node20:ro,rshared'
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
steps:
- name: install nodejs20glibc2.17
run: |
yum install curl -yyq
curl -LO https://unofficial-builds.nodejs.org/download/release/v20.9.0/node-v20.9.0-linux-x64-glibc-217.tar.xz
tar -xf node-v20.9.0-linux-x64-glibc-217.tar.xz --strip-components 1 -C /node20217
ldd /__e/node20/bin/node

- name: Checkout Sysdig
uses: actions/checkout@v4
with:
path: sysdig
- name: Link paths
run: |
mkdir -p /source
ln -s "$GITHUB_WORKSPACE/sysdig" /source/sysdig
- name: Restore build skeleton
id: cache
uses: actions/cache/restore@v3
with:
path: /build-skeleton
key: build-skeleton-${{ github.run_id }}
restore-keys: build-skeleton-
- name: Build
run: build cmake
- name: Build packages
run: build package
- name: Upload rpm package
uses: actions/upload-artifact@v3
with:
name: sysdig-dev-linux-x86_64.rpm
path: |
/build/release/sysdig-*.rpm
- name: Upload deb package
uses: actions/upload-artifact@v3
with:
name: sysdig-dev-linux-x86_64.deb
path: |
/build/release/sysdig-*.deb
- name: Upload tar.gz package
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: sysdig-dev-linux-x86_64.tar.gz
name: ${{ steps.artifact_name.outputs.name }}.rpm
path: |
/build/release/sysdig-*.tar.gz
build/sysdig-*.rpm

build-sysdig-linux-arm64:
needs: builder
env:
REGISTRY: ghcr.io
BUILDER_IMAGE_BASE: ghcr.io/draios/sysdig-builder-pr
SKEL_BUILDER_IMAGE_BASE: ghcr.io/draios/sysdig-skel-builder-pr
BUILDER_DEV: ghcr.io/draios/sysdig-builder:dev
SKEL_BUILDER_DEV: ghcr.io/draios/sysdig-skel-builder:dev
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
runs-on: ubuntu-latest
steps:
- name: Checkout Sysdig
uses: actions/checkout@v4
with:
path: sysdig
- name: Create build dir
run: |
mkdir -p ${{ github.workspace }}/sysdig-build-aarch64

- name: Login to Github Packages
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: 'amd64,arm64'
- name: Run the build skeleton process with Docker
uses: addnab/docker-run-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ env.REGISTRY }}
image: ${{ needs.builder.outputs.skeleton_builder_image }}
options: --platform=linux/arm64 -v ${{ github.workspace }}/sysdig:/source/sysdig -v ${{ github.workspace }}/sysdig-build-aarch64:/build/dev-packages -v ${{ github.workspace }}/skeleton-build:/build-skeleton
run: |
mkdir -p /build/dev-packages && \
build cmake-skeleton && \
build make-skeleton
- name: Run the build process with Docker
uses: addnab/docker-run-action@v3
with:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ env.REGISTRY }}
image: ${{ needs.builder.outputs.builder_image }}
options: --platform=linux/arm64 -v ${{ github.workspace }}/sysdig:/source/sysdig -v ${{ github.workspace }}/sysdig-build-aarch64:/build/dev-packages -v ${{ github.workspace }}/skeleton-build:/build-skeleton
run: |
mkdir -p /build/dev-packages && \
build cmake && \
build package && \
cp /build/release/sysdig-* /build/dev-packages
- name: Upload deb package
uses: actions/upload-artifact@v3
with:
name: sysdig-dev-linux-aarch64.deb
path: |
${{ github.workspace }}/sysdig-build-aarch64/sysdig-*.deb
- name: Upload rpm package
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: sysdig-dev-linux-aarch64.rpm
name: ${{ steps.artifact_name.outputs.name }}.deb
path: |
${{ github.workspace }}/sysdig-build-aarch64/sysdig-*.rpm
build/sysdig-*.deb

- name: Upload tar.gz package
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: sysdig-dev-linux-aarch64.tar.gz
name: ${{ steps.artifact_name.outputs.name }}.tar.gz
path: |
${{ github.workspace }}/sysdig-build-aarch64/sysdig-*.tar.gz
build/sysdig-*.tar.gz

build-sysdig-others-amd64:
name: build-sysdig-other-amd64
Expand All @@ -248,8 +114,6 @@
os: [windows-latest, macos-13]
include:
- os: windows-latest
artifact_name: win
artifact_ext: exe
- os: macos-13
artifact_name: osx
artifact_ext: dmg
Expand All @@ -259,11 +123,10 @@
uses: actions/checkout@v4
- name: Build
run: |
mkdir -p build
cd build && cmake -Wno-dev ..
cmake --build . --target package --config Release
cmake -Wno-dev -S . -B build
cmake --build build --target package --config Release
- name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: sysdig-dev-${{ matrix.artifact_name }}-x86_64.${{ matrix.artifact_ext }}
path: |
Expand All @@ -284,11 +147,10 @@
uses: actions/checkout@v4
- name: Build
run: |
mkdir -p build
cd build && cmake -Wno-dev ..
cmake --build . --target package --config Release
cmake -Wno-dev -S . -B build
cmake --build build --target package --config Release
- name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: sysdig-dev-${{ matrix.artifact_name }}-arm64.${{ matrix.artifact_ext }}
path: |
Expand Down
Loading
Loading