Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: install mkosi from repository #2233

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions .github/workflows/azure-podvm-image-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,29 @@ jobs:
path: cloud-api-adaptor
ref: "${{ inputs.git-ref || 'main' }}"

- uses: cachix/install-nix-action@v30
with:
nix_path: nixpkgs=channel:nixos-unstable
- name: Install build dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y \
alien \
bubblewrap \
dnf \
mtools \
qemu-utils \
systemd-ukify \
uidmap
sudo snap install yq

- name: Read properties from versions.yaml
run: |
echo "MKOSI_VERSION=$(yq -e '.tools.mkosi' versions.yaml)" >> "$GITHUB_ENV"

- name: Setup mkosi
run: |
git clone -b "$MKOSI_VERSION" https://github.com/systemd/mkosi
PATH="$PWD/mkosi/bin:$PATH"
mkosi --version
echo "PATH=$PWD/mkosi/bin:$PATH" >> "$GITHUB_ENV"

- name: Install uplosi
run: |
Expand All @@ -67,11 +87,6 @@ jobs:
run: |
make binaries

- name: Install build dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y qemu-utils

- name: Build image
run: make image

Expand Down
35 changes: 15 additions & 20 deletions .github/workflows/podvm_mkosi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,17 +114,30 @@ jobs:
- name: Install build dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y bubblewrap alien dnf qemu-utils uidmap
sudo apt-get install -y \
alien \
bubblewrap \
dnf \
mtools \
qemu-utils \
systemd-ukify \
uidmap
sudo snap install yq

- name: Read properties from versions.yaml
run: |
echo "MKOSI_VERSION=$(yq -e '.tools.mkosi' versions.yaml)" >> "$GITHUB_ENV"
echo "ORAS_VERSION=$(yq -e '.tools.oras' versions.yaml)" >> "$GITHUB_ENV"

- uses: oras-project/setup-oras@v1
with:
version: ${{ env.ORAS_VERSION }}

- name: Setup mkosi
run: |
git clone -b "$MKOSI_VERSION" https://github.com/systemd/mkosi
PATH="$PWD/mkosi/bin:$PATH"
mkosi --version
echo "PATH=$PWD/mkosi/bin:$PATH" >> "$GITHUB_ENV"

- name: Build binaries
id: build_binaries
Expand All @@ -133,22 +146,6 @@ jobs:
env:
ARCH: ${{ inputs.arch }}

- name: Install mkosi
if: ${{ inputs.arch == 's390x' }}
run: |
git clone -b v22 https://github.com/systemd/mkosi
sudo rm -f /usr/local/bin/mkosi
sudo ln -s "$PWD/mkosi/bin/mkosi" /usr/local/bin/mkosi
mkosi --version

- name: Install Nix
if: ${{ inputs.arch == 'amd64' }}
uses: cachix/install-nix-action@v30

- name: Build nix shell to cache dependencies
if: ${{ inputs.arch == 'amd64' }}
run: nix build .#devShells.x86_64-linux.podvm-mkosi

- name: Build mkosi debug image
if: ${{ inputs.debug == 'true' }}
working-directory: src/cloud-api-adaptor/podvm-mkosi
Expand Down Expand Up @@ -192,13 +189,11 @@ jobs:
subject-digest: ${{ steps.publish_oras_qcow2.outputs.digest }}
push-to-registry: true


- name: Clean up some space for the docker provider build
working-directory: src/cloud-api-adaptor/podvm-mkosi
run: |
mkulke marked this conversation as resolved.
Show resolved Hide resolved
sudo du --max-depth=2 /home/runner || true
sudo du --max-depth=2 /var/lib || true
sudo rm -rf /nix
sudo rm -rf ./build
sudo rm -rf ./mkosi.cache

Expand Down
61 changes: 0 additions & 61 deletions src/cloud-api-adaptor/flake.lock

This file was deleted.

45 changes: 0 additions & 45 deletions src/cloud-api-adaptor/flake.nix

This file was deleted.

11 changes: 2 additions & 9 deletions src/cloud-api-adaptor/podvm-mkosi/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,37 +65,30 @@ image:
rm -rf ./build
@echo "Building image..."
ifeq ($(SE_BOOT),true)
touch resources/buildS390xImage
sudo mkosi --profile production.conf --image system
sudo -E ../hack/build-s390x-se-image.sh
else ifeq ($(ARCH),s390x)
touch resources/buildS390xImage
sudo mkosi --profile production.conf --image system
sudo -E ../hack/build-s390x-image.sh
else
touch resources/buildBootableImage
nix develop ..#podvm-mkosi --command mkosi --environment=VARIANT_ID=production
mkosi --profile production.conf
qemu-img convert -f raw -O qcow2 build/system.raw build/podvm-$(PODVM_DISTRO)-$(ARCH).qcow2
endif

PHONY: image-debug
image-debug:
@echo "Enabling debug preset..."
rm -rf resources/build*Image
touch resources/buildDebugImage
rm -rf ./build
mkulke marked this conversation as resolved.
Show resolved Hide resolved
@echo "Building debug image..."
ifeq ($(SE_BOOT),true)
touch resources/buildS390xImage
sudo mkosi --profile debug.conf
sudo -E ../hack/build-s390x-se-image.sh
else ifeq ($(ARCH),s390x)
touch resources/buildS390xImage
sudo mkosi --profile debug.conf
sudo -E ../hack/build-s390x-image.sh
else
touch resources/buildBootableImage
nix develop ..#podvm-mkosi --command mkosi --environment=VARIANT_ID=debug
mkosi --profile debug.conf
qemu-img convert -f raw -O qcow2 build/system.raw build/podvm-$(PODVM_DISTRO)-$(ARCH).qcow2
endif

Expand Down
6 changes: 4 additions & 2 deletions src/cloud-api-adaptor/podvm-mkosi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

## Prerequisites

Currently, mksoi and other related tools are provided through a [Nix](https://nixos.org/) flake. Nix ensures all tools used in the build of the image are itself reproducible and pinned. mkosi requires a very recent systemd version, so using tools installed on the host is usually not possible. Nix needs to be configured to enable `flakes` and `nix command`. It is recommended to install Nix with the `DeterminateSystems nix-installer`, which comes with a configuration that is ready to use.
There are various ways to install mkosi documented on the [project page](https://github.com/systemd/mkosi). Different distributions also package mkosi in their repositories, alongside the dependencies.

Refer to the [CI workflow](../../../.github/workflows/podvm_mkosi.yaml) to see which additional tools are required to build an image.

### Building the image

Expand Down Expand Up @@ -77,7 +79,7 @@ reduce complexity of configuration and CI and shall not be seen as open to-dos.
from IMDS via the project's `process-user-data` tool.

## Build s390x image
Since the [nix OS](https://nixos.org/download/#download-nix) does not support s390x, we can use the mkosi **ToolsTree** feature defined in `mkosi.conf` to download latest tools automatically:
We can use the mkosi **ToolsTree** feature defined in `mkosi.conf` to download latest tools automatically:
wainersm marked this conversation as resolved.
Show resolved Hide resolved
```
[Host]
ToolsTree=default
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
[Match]
Distribution=fedora

# mkosi version in nix is 17.1,
# which doesn't support Architecture in [Match]
# As a workaround, use a flag file instead.
#Architecture=s390x
PathExists=../../resources/buildS390xImage
Architecture=s390x

[Content]
Packages=kernel-core
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
[Match]
Distribution=fedora

# mkosi version in nix is 17.1,
# which doesn't support Architecture in [Match]
# As a workaround, use a flag file instead.
#Architecture=!s390x
PathExists=../../resources/buildBootableImage
Architecture=!s390x

[Content]
Packages=systemd-boot
Original file line number Diff line number Diff line change
@@ -1,12 +1,9 @@
[Match]
Distribution=fedora

PathExists=../../resources/buildDebugImage

Profile=debug
# Overwrite default ssh config, but conflict with
# cloud-init which is installed for s390x.
#Architecture=!s390x
PathExists=../../resources/buildBootableImage
Architecture=!s390x

[Content]
ExtraTrees=../../mkosi.skeleton-debug
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
[Match]
Distribution=fedora
# Only for debug images and if authorized_keys exists
PathExists=../../resources/buildDebugImage
Profile=debug
PathExists=../../resources/authorized_keys

[Content]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[Match]
Distribution=fedora
# This is a little hack to define different image types in a mkosi config.
# There is also imageId, but it renames the output, which is not what we want.
PathExists=../../resources/buildDebugImage
Profile=debug

[Content]
Autologin=true
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
[Match]
Distribution=fedora

# mkosi version in nix is 17.1,
# which doesn't support Architecture in [Match]
# As a workaround, use a flag file instead.
#Architecture=s390x
PathExists=../../resources/buildS390xImage
Architecture=s390x

[Content]
Bootable=no
Expand Down
5 changes: 3 additions & 2 deletions src/cloud-api-adaptor/versions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,12 @@ cloudimg:

tools:
bats: 1.10.0
iptables-wrapper: v0.0.0-20240819165702-06cad2ec6cb5
golang: 1.22.7
kcli: 99.0.202408152044
mkosi: v22
protoc: 3.15.0
packer: v1.9.4
kcli: 99.0.202408152044
iptables-wrapper: v0.0.0-20240819165702-06cad2ec6cb5
oras: 1.2.0
# Referenced Git repositories
git:
Expand Down
Loading