Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: automatically build kernels #1258

Closed
3 tasks done
lmb opened this issue Dec 6, 2023 · 0 comments · Fixed by #1264
Closed
3 tasks done

CI: automatically build kernels #1258

lmb opened this issue Dec 6, 2023 · 0 comments · Fixed by #1264
Assignees
Labels
help wanted Extra attention is needed

Comments

@lmb
Copy link
Collaborator

lmb commented Dec 6, 2023

CI relies on pre built kernels for testing. Those kernels live in the https://github.com/cilium/ci-kernels repository and are stored in large file storage. Right now adding a new kernel is a manual process:

  1. Edit make.sh
  2. Run make
  3. Commit the resulting .tgz into the repo
  4. Push, PR, merge

We should automate this and store the result as a container image on ghcr.io. The current build process already uses Docker behind the scenes, with the appropriate image targets.

Rough idea for a workflow:

  1. Edit make.sh
  2. Commit, Push
  3. GHA builds kernel via existing Dockerfiles
  4. Merge
  5. GHA builds and pushes images from main (vmlinux, selftests)

This will require modifying run-tests.sh to pull kernel + selftests from ghcr.io instead of the .tgz. @tb says that containers are just tars, so maybe we can get away with some additional bash magic to go from an image url to a tgz we can use. If that is too complicated we can maybe reuse docker somehow to just do an image download for us?

This new workflow has one downside: images created from a branch build are not available anywhere, so executing ./run-tests.sh against them isn't straight forward. One solution I can think of is to just move the build tooling into the ebpf repo itself. ci-kernels really exists to hold the .tgz so once we stop doing that there is no need to keep it around anymore. We already have docker stuff in testdata/docker after all.

Tasks

Preview Give feedback
@lmb lmb added the help wanted Extra attention is needed label Dec 6, 2023
@lmb lmb changed the title CI: automatically build ci-kernels CI: automatically build kernels Dec 6, 2023
@lmb lmb self-assigned this Dec 8, 2023
lmb added a commit to lmb/ebpf that referenced this issue Dec 8, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: cilium#1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
lmb added a commit to lmb/ebpf that referenced this issue Dec 8, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: cilium#1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
lmb added a commit to lmb/ebpf that referenced this issue Dec 13, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: cilium#1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
lmb added a commit to lmb/ebpf that referenced this issue Dec 13, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: cilium#1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
lmb added a commit to lmb/ebpf that referenced this issue Dec 14, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: cilium#1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb lmb closed this as completed in #1264 Dec 14, 2023
lmb added a commit that referenced this issue Dec 14, 2023
Use the new GH Actions built containers as a source for run-tests.sh.

Fixes: #1258

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb lmb reopened this Dec 14, 2023
@lmb lmb closed this as completed Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant