Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RSDK-3154] Switch to BuildJet for arm64 action runners #2387

Merged
merged 22 commits into from
May 19, 2023
Merged

[RSDK-3154] Switch to BuildJet for arm64 action runners #2387

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1e222b4
Test buildjet speeds
Otterverse May 17, 2023
5cf2208
Fix and test
Otterverse May 17, 2023
3959f54
Fix and test
Otterverse May 17, 2023
3449884
Fix and test
Otterverse May 17, 2023
7f6bef4
Fix and test
Otterverse May 17, 2023
1f943b8
Fix and test
Otterverse May 17, 2023
34ba4ea
AppImage Only
Otterverse May 17, 2023
71507f3
AppImage Only
Otterverse May 17, 2023
15e373f
AppImage Only
Otterverse May 17, 2023
e0e4903
AppImage Only
Otterverse May 17, 2023
a1aa3c5
Test BuildJet
Otterverse May 18, 2023
7722508
Test BuildJet
Otterverse May 18, 2023
4750c63
Test BuildJet
Otterverse May 19, 2023
2b168f3
Test BuildJet
Otterverse May 19, 2023
342801b
Test BuildJet
Otterverse May 19, 2023
062b0f7
Test BuildJet
Otterverse May 19, 2023
a5b7ee2
Test BuildJet
Otterverse May 19, 2023
53a8603
Test BuildJet
Otterverse May 19, 2023
11f0770
Test BuildJet
Otterverse May 19, 2023
ea0df7e
Test BuildJet
Otterverse May 19, 2023
9287421
DeSMURF
Otterverse May 19, 2023
175ce31
Disable coverage on arm64
Otterverse May 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/actionlint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,13 @@ self-hosted-runner:
labels:
- qemu-host
- pi4
- buildjet-2vcpu-ubuntu-2204
- buildjet-4vcpu-ubuntu-2204
- buildjet-8vcpu-ubuntu-2204
- buildjet-16vcpu-ubuntu-2204
- buildjet-32vcpu-ubuntu-2204
- buildjet-2vcpu-ubuntu-2204-arm
- buildjet-4vcpu-ubuntu-2204-arm
- buildjet-8vcpu-ubuntu-2204-arm
- buildjet-16vcpu-ubuntu-2204-arm
- buildjet-32vcpu-ubuntu-2204-arm
16 changes: 11 additions & 5 deletions .github/workflows/appimage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: Build AppImage

on:
workflow_dispatch:
inputs:
release_type:
required: true
type: string
workflow_call:
inputs:
release_type:
Expand All @@ -15,12 +19,13 @@ jobs:
appimage:
name: AppImage Build
strategy:
fail-fast: false
matrix:
include:
- arch: [x64, qemu-host]
image: ghcr.io/viamrobotics/canon:amd64-cache
platform: linux/amd64
- arch: [arm64, qemu-host]
- arch: [buildjet-8vcpu-ubuntu-2204-arm]
image: ghcr.io/viamrobotics/canon:arm64-cache
platform: linux/arm64
runs-on: ${{ matrix.arch }}
Expand All @@ -35,15 +40,12 @@ jobs:
- name: Check out code
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2

- name: Clean and Fix Permissions
run: |
Expand Down Expand Up @@ -114,8 +116,9 @@ jobs:
name: AppImage Test
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push'
strategy:
fail-fast: false
matrix:
arch: [[x64, qemu-host], [arm64, qemu-host], [arm64, pi4]]
arch: [[x64, qemu-host], [buildjet-2vcpu-ubuntu-2204-arm], [arm64, pi4]]
needs: appimage
runs-on: ${{ matrix.arch }}
timeout-minutes: 15
Expand All @@ -131,6 +134,9 @@ jobs:

- name: Test AppImage
run: |
if echo "${{ matrix.arch[0] }}" | grep -q buildjet; then
sudo apt-get install -y libfuse2
fi
channel="${{ github.ref_name }}"
# we call our main branch releases "latest"
if [ "$channel" = "main" ]; then
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
image: ghcr.io/viamrobotics/canon:amd64
platform: linux/amd64
make_target: 'canon-cache-amd64-ci'
- arch: [arm64, qemu-host]
- arch: [buildjet-8vcpu-ubuntu-2204-arm]
image: ghcr.io/viamrobotics/canon:arm64
platform: linux/arm64
make_target: 'canon-cache-arm64-ci'
Expand Down Expand Up @@ -60,7 +60,7 @@ jobs:
image: ghcr.io/viamrobotics/canon:amd64
platform: linux/amd64
make_target: 'antique-cache-amd64-ci'
- arch: [arm64, qemu-host]
- arch: [buildjet-8vcpu-ubuntu-2204-arm]
image: ghcr.io/viamrobotics/canon:arm64
platform: linux/arm64
make_target: 'antique-cache-arm64-ci'
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/motion-benchmarks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,12 @@ jobs:
- name: Check out main branch code
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2

- name: Check out code in motion-testing repository
uses: actions/checkout@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/npm-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
jobs:
publish:
if: github.repository_owner == 'viamrobotics'
runs-on: [self-hosted, x64]
runs-on: [x64, qemu-host]
container:
image: ghcr.io/viamrobotics/canon:amd64-cache

Expand Down
13 changes: 8 additions & 5 deletions .github/workflows/staticbuild.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: Build Semi-Static Binary

on:
workflow_dispatch:
inputs:
release_type:
required: true
type: string
workflow_call:
inputs:
release_type:
Expand All @@ -15,12 +19,13 @@ jobs:
static:
name: Antique Build
strategy:
fail-fast: false
matrix:
include:
- arch: [x64, qemu-host]
image: ghcr.io/viamrobotics/antique:amd64-cache
platform: linux/amd64
- arch: [arm64, qemu-host]
- arch: [buildjet-8vcpu-ubuntu-2204-arm]
image: ghcr.io/viamrobotics/antique:arm64-cache
platform: linux/arm64
runs-on: ${{ matrix.arch }}
Expand All @@ -35,15 +40,12 @@ jobs:
- name: Check out code
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2

- name: Clean and Fix Permissions
run: |
Expand Down Expand Up @@ -114,8 +116,9 @@ jobs:
name: Static Test
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push'
strategy:
fail-fast: false
matrix:
arch: [[x64, qemu-host], [arm64, qemu-host], [arm64, pi4]]
arch: [[x64, qemu-host], [buildjet-2vcpu-ubuntu-2204-arm], [arm64, pi4]]
needs: static
runs-on: ${{ matrix.arch }}
timeout-minutes: 15
Expand Down
22 changes: 7 additions & 15 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,17 @@ jobs:
build_and_test:
name: Build and Test
strategy:
fail-fast: false
matrix:
include:
- arch: [x64, qemu-host]
image: ghcr.io/viamrobotics/canon:amd64-cache
platform: linux/amd64
build_lint: 'make build lint'
- arch: [arm64, qemu-host]
test_cmd: "make cover test-web"
- arch: [buildjet-8vcpu-ubuntu-2204-arm]
image: ghcr.io/viamrobotics/canon:arm64-cache
platform: linux/arm64
build_lint: 'make build-go lint-go'
test_cmd: "make test-go test-web"
runs-on: ${{ matrix.arch }}
container:
image: ${{ matrix.image }}
Expand All @@ -34,8 +35,6 @@ jobs:
- name: Check out code
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
Expand All @@ -60,12 +59,12 @@ jobs:
echo "GITHUB_X_PR_BASE_SHA=${{ github.event.pull_request.base.sha }}" >> $GITHUB_ENV
echo "GITHUB_X_PR_BASE_REF=${{ github.event.pull_request.base.ref }}" >> $GITHUB_ENV

- name: Verify no uncommitted changes from "${{ matrix.build_lint }}"
- name: Verify no uncommitted changes from "make build lint"
run: |
git init
git add .
chown -R testbot:testbot .
sudo -Hu testbot bash -lc '${{ matrix.build_lint }}'
sudo -Hu testbot bash -lc 'make build lint'
GEN_DIFF=$(git status -s)

if [ -n "$GEN_DIFF" ]; then
Expand All @@ -75,9 +74,8 @@ jobs:
fi

- name: Test
if: matrix.platform == 'linux/amd64'
run: |
sudo --preserve-env=MONGODB_TEST_OUTPUT_URI,GITHUB_SHA,GITHUB_RUN_ID,GITHUB_RUN_NUMBER,GITHUB_RUN_ATTEMPT,GITHUB_X_PR_BASE_SHA,GITHUB_X_PR_BASE_REF,GITHUB_X_HEAD_REF,GITHUB_X_HEAD_SHA,GITHUB_REPOSITORY -Hu testbot bash -lc 'make cover test-web'
sudo --preserve-env=MONGODB_TEST_OUTPUT_URI,GITHUB_SHA,GITHUB_RUN_ID,GITHUB_RUN_NUMBER,GITHUB_RUN_ATTEMPT,GITHUB_X_PR_BASE_SHA,GITHUB_X_PR_BASE_REF,GITHUB_X_HEAD_REF,GITHUB_X_HEAD_SHA,GITHUB_REPOSITORY -Hu testbot bash -lc '${{ matrix.test_cmd }}'

# Now that RDK is public, can't directly comment without token having full read/write access
# code-coverage-comment.yml will trigger seperately and post the actual comments
Expand Down Expand Up @@ -120,15 +118,12 @@ jobs:
- name: Check out main branch code
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you check if this affects coverage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added "test_cmd" to the Test step so amd64 runs cover/test-web, and arm64 does test-go/test-web instead.

fetch-depth: 2

- name: Test
run: make test-pi
Expand All @@ -143,15 +138,12 @@ jobs:
- name: Check out main branch code
if: github.event_name != 'pull_request_target'
uses: actions/checkout@v3
with:
fetch-depth: 2

- name: Check out PR branch code
if: github.event_name == 'pull_request_target'
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 2

- name: Install dependencies
run: |
Expand Down
5 changes: 3 additions & 2 deletions components/board/pi/impl/board_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ package piimpl

import (
"context"
"os"
"testing"
"time"

Expand All @@ -30,8 +31,8 @@ func TestPiPigpio(t *testing.T) {
resourceConfig := resource.Config{ConvertedAttributes: &cfg}

pp, err := newPigpio(ctx, board.Named("foo"), resourceConfig, logger)
if err != nil && err.Error() == "not running on a pi" {
t.Skip("not running on a pi")
if os.Getuid() != 0 || err != nil && err.Error() == "not running on a pi" {
t.Skip("not running as root on a pi")
return
}
test.That(t, err, test.ShouldBeNil)
Expand Down
5 changes: 3 additions & 2 deletions components/board/pi/impl/external_hardware_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ package piimpl

import (
"context"
"os"
"testing"
"time"

Expand Down Expand Up @@ -40,8 +41,8 @@ func TestPiHardware(t *testing.T) {
resourceConfig := resource.Config{ConvertedAttributes: &cfg}

pp, err := newPigpio(ctx, board.Named("foo"), resourceConfig, logger)
if err != nil && err.Error() == "not running on a pi" {
t.Skip("not running on a pi")
if os.Getuid() != 0 || err != nil && err.Error() == "not running on a pi" {
t.Skip("not running as root on a pi")
return
}
test.That(t, err, test.ShouldBeNil)
Expand Down