Integrate CRI-O into `krel obs` #3214

saschagrunert · 2023-08-21T10:20:49Z

The CRI-O project wants to simplify it's packaging process and integrate it tighter into the Kubernetes infrastructure. CRI-O already uses OBS for the various supported release branches, where everything is built from source.

Goal for 1.29 is to integrate CRI-O into k/release by using the same pattern as other tools like cri-tools use together with krel obs.

Useful information

CRI-O already produces static binary bundles for amd64, arm64 and ppc64le
(s390x support is blocked by pkgsCross.s390.stdenv broken by #238154 NixOS/nixpkgs#243614)
The bundle contains all runtime dependencies which are required (and optional) to run CRI-O, means we can omit some of its content (like the cni-plugins, make that a requirement in the spec).
We probably need to change the CRI-O release cycle to release ahead of Kubernetes like we did in:
Release ahead of Kubernetes kubernetes-sigs/cri-tools#1213 (cc @haircommander)

The text was updated successfully, but these errors were encountered:

haircommander · 2023-08-22T15:10:51Z

+1 from me. In my ideal we wouldn't have to release before kubernetes but that's not a hard blocker for me

saschagrunert · 2023-08-23T07:02:58Z

+1 from me. In my ideal we wouldn't have to release before kubernetes but that's not a hard blocker for me

I think we can also release afterwards if we do not make the kubelet dependent on CRI-O, which we would probably not do in any case.

saschagrunert · 2023-09-13T11:22:09Z

Successfully pushed v1.28.1 via:

> go run ./cmd/krel obs stage --architectures amd64,arm64,ppc64le --packages cri-o --version 1.28.1 --project isv:kubernetes:core:shared:build --nomock --stream

https://build.opensuse.org/package/show/isv:kubernetes:core:shared:build/cri-o

saschagrunert · 2023-09-13T12:49:45Z

Another run containing #3268 and #3269:

> go run ./cmd/krel obs stage --architectures amd64,arm64,ppc64le --packages cri-o --version 1.28.1 --project isv:kubernetes:core:shared:build --nomock --stream

Logs: https://console.cloud.google.com/cloud-build/builds/e3b3d258-3ed3-4748-832a-b1cf38590867?project=648026197307
Results: https://build.opensuse.org/project/show/isv:kubernetes:core:shared:build

Test of the `rpm`

> podman run -it opensuse/tumbleweed sh -c \
    'zypper addrepo https://download.opensuse.org/repositories/isv:kubernetes:core:shared:build/rpm/isv:kubernetes:core:shared:build.repo && \
     zypper refresh && \
     zypper install -y cri-o bash && \
     crio -l debug -s vfs'
…
INFO[2023-09-13 12:49:05.730998887Z] Starting CRI-O, version: 1.28.1, git: eda470f7f503d9f40a9aa2a02e45f0878ed6fc61(dirty)
INFO[2023-09-13 12:49:05.733994040Z] Node configuration value for hugetlb cgroup is false
INFO[2023-09-13 12:49:05.734001328Z] Node configuration value for pid cgroup is true
INFO[2023-09-13 12:49:05.734021773Z] Node configuration value for memoryswap cgroup is true
INFO[2023-09-13 12:49:05.734025245Z] Node configuration value for cgroup v2 is true
WARN[2023-09-13 12:49:05.734063312Z] node configuration validation for systemd CollectMode failed: check systemd CollectMode: exec: "systemctl": executable file not found in $PATH
INFO[2023-09-13 12:49:05.734067382Z] Node configuration value for systemd CollectMode is false
WARN[2023-09-13 12:49:05.734078175Z] node configuration validation for systemd AllowedCPUs failed: check systemd AllowedCPUs: exec: "systemctl": executable file not found in $PATH
INFO[2023-09-13 12:49:05.734081250Z] Node configuration value for systemd AllowedCPUs is false
DEBU[2023-09-13 12:49:05.734267042Z] [graphdriver] trying provided driver "vfs"    file="drivers/driver.go:359"
INFO[2023-09-13 12:49:05.734448016Z] Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL  file="capabilities/capabilities_linux.go:38"
WARN[2023-09-13 12:49:05.734489674Z] 'runc is being ignored due to: "\"runc\" not found in $PATH: exec: \"runc\": executable file not found in $PATH"  file="config/config.go:1220"
DEBU[2023-09-13 12:49:05.734515815Z] Using runtime executable from $PATH "/usr/bin/crun"  file="config/config.go:1492"
DEBU[2023-09-13 12:49:05.734528586Z] Found valid runtime "crun" for runtime_path "/usr/bin/crun"  file="config/config.go:1504"
DEBU[2023-09-13 12:49:05.734545828Z] Allowed annotations for runtime: [io.containers.trace-syscall]  file="config/config.go:1539"
ERRO[2023-09-13 12:49:05.736602599Z] Getting OCI runtime features failed: exit status 1  file="config/config.go:1239"
DEBU[2023-09-13 12:49:05.736701604Z] Loading registries configuration "/etc/containers/registries.conf"  file="sysregistriesv2/system_registries_v2.go:926"
DEBU[2023-09-13 12:49:05.736926941Z] Using hooks directory: /usr/share/containers/oci/hooks.d  file="config/config.go:1109"
DEBU[2023-09-13 12:49:05.737096792Z] Using pinns from $PATH: /usr/bin/pinns        file="config/config.go:1379"
INFO[2023-09-13 12:49:05.737261971Z] Checkpoint/restore support disabled           file="config/config.go:1134"
INFO[2023-09-13 12:49:05.737287773Z] Using seccomp default profile when unspecified: true  file="seccomp/seccomp.go:99"
INFO[2023-09-13 12:49:05.737324311Z] Using the internal default seccomp profile    file="seccomp/seccomp.go:152"
INFO[2023-09-13 12:49:05.737352846Z] AppArmor is disabled by the system or at CRI-O build-time  file="apparmor/apparmor_linux.go:34"
INFO[2023-09-13 12:49:05.737379693Z] No blockio config file specified, blockio not configured  file="blockio/blockio.go:37"
INFO[2023-09-13 12:49:05.737407247Z] RDT not available in the host system          file="rdt/rdt.go:56"
DEBU[2023-09-13 12:49:05.737452799Z] Using conmon from $PATH: /usr/bin/conmon      file="config/config.go:1379"
INFO[2023-09-13 12:49:05.738130522Z] Conmon does support the --sync option         file="conmonmgr/conmonmgr.go:85"
INFO[2023-09-13 12:49:05.738165968Z] Conmon does support the --log-global-size-max option  file="conmonmgr/conmonmgr.go:71"
INFO[2023-09-13 12:49:05.739518914Z] Found CNI network crio (type=bridge) at /etc/cni/net.d/11-crio-ipv4-bridge.conflist  file="ocicni/ocicni.go:343"
INFO[2023-09-13 12:49:05.739549533Z] Updated default CNI network name to crio      file="ocicni/ocicni.go:375"
DEBU[2023-09-13 12:49:05.739816410Z] reading hooks from /usr/share/containers/oci/hooks.d  file="hooks/read.go:65"
INFO[2023-09-13 12:49:05.739870801Z] Attempting to restore irqbalance config from /etc/sysconfig/orig_irq_banned_cpus  file="server/server.go:430"
INFO[2023-09-13 12:49:05.739915328Z] Restore irqbalance config: failed to get current CPU ban list, ignoring  file="runtimehandlerhooks/high_performance_hooks.go:565"
DEBU[2023-09-13 12:49:05.741566019Z] Golang's threads limit set to 460710          file="server/server.go:380"
WARN[2023-09-13 12:49:05.741644202Z] Error encountered when checking whether cri-o should wipe containers: open /var/run/crio/version: no such file or directory  file="server/server.go:640"
DEBU[2023-09-13 12:49:05.741742447Z] Sandboxes: []                                 file="server/server.go:545"
INFO[2023-09-13 12:49:05.741775977Z] Registered SIGHUP reload watcher              file="server/server.go:597"
DEBU[2023-09-13 12:49:05.741804263Z] Metrics are disabled                          file="server/server.go:555"
INFO[2023-09-13 12:49:05.741832799Z] Starting seccomp notifier watcher             file="server/server.go:758"
INFO[2023-09-13 12:49:05.741906815Z] Create NRI interface                          file="nri/nri.go:94"
INFO[2023-09-13 12:49:05.741925101Z] NRI interface is disabled in the configuration.  file="nri/nri.go:101"
DEBU[2023-09-13 12:49:05.762014227Z] monitoring "/usr/share/containers/oci/hooks.d" for hooks  file="hooks/monitor.go:43"

Test of the `deb`

> podman run -it debian:12 sh -c \
    'apt-get update && \
     apt-get install -y bash curl software-properties-common && \
     (echo \'deb http://download.opensuse.org/repositories/isv:/kubernetes:/core:/shared:/build/deb/ /\' | tee /etc/apt/sources.list.d/isv:kubernetes:core:shared:build.list) && \
     (curl -fsSL https://download.opensuse.org/repositories/isv:kubernetes:core:shared:build/deb/Release.key | gpg --dearmor | tee /etc/apt/trusted.gpg.d/isv_kubernetes_core_shared_build.gpg > /dev/null) && \
     apt-get update && \
     apt-get install -y cri-o && \
     crio -l debug -s vfs'
…
INFO[2023-09-13 12:49:00.071704285Z] Starting CRI-O, version: 1.28.1, git: eda470f7f503d9f40a9aa2a02e45f0878ed6fc61(dirty)
INFO[2023-09-13 12:49:00.074795097Z] Node configuration value for hugetlb cgroup is false
INFO[2023-09-13 12:49:00.074806176Z] Node configuration value for pid cgroup is true
INFO[2023-09-13 12:49:00.074832469Z] Node configuration value for memoryswap cgroup is true
INFO[2023-09-13 12:49:00.074836118Z] Node configuration value for cgroup v2 is true
WARN[2023-09-13 12:49:00.078287172Z] node configuration validation for systemd CollectMode failed: check systemd CollectMode: exit status 1
INFO[2023-09-13 12:49:00.078302017Z] Node configuration value for systemd CollectMode is false
WARN[2023-09-13 12:49:00.080261932Z] node configuration validation for systemd AllowedCPUs failed: check systemd AllowedCPUs: exit status 1
INFO[2023-09-13 12:49:00.080275770Z] Node configuration value for systemd AllowedCPUs is false
DEBU[2023-09-13 12:49:00.080558079Z] [graphdriver] trying provided driver "vfs"    file="drivers/driver.go:359"
INFO[2023-09-13 12:49:00.080821914Z] Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL  file="capabilities/capabilities_linux.go:38"
WARN[2023-09-13 12:49:00.080892219Z] 'runc is being ignored due to: "\"runc\" not found in $PATH: exec: \"runc\": executable file not found in $PATH"  file="config/config.go:1220"
DEBU[2023-09-13 12:49:00.080953879Z] Using runtime executable from $PATH "/usr/bin/crun"  file="config/config.go:1492"
DEBU[2023-09-13 12:49:00.080982065Z] Found valid runtime "crun" for runtime_path "/usr/bin/crun"  file="config/config.go:1504"
DEBU[2023-09-13 12:49:00.081015091Z] Allowed annotations for runtime: [io.containers.trace-syscall]  file="config/config.go:1539"
ERRO[2023-09-13 12:49:00.082322444Z] Getting OCI runtime features failed: exit status 1  file="config/config.go:1239"
DEBU[2023-09-13 12:49:00.082348782Z] Loading registries configuration "/etc/containers/registries.conf"  file="sysregistriesv2/system_registries_v2.go:926"
DEBU[2023-09-13 12:49:00.082495862Z] Using hooks directory: /usr/share/containers/oci/hooks.d  file="config/config.go:1109"
DEBU[2023-09-13 12:49:00.082610290Z] Using pinns from $PATH: /usr/bin/pinns        file="config/config.go:1379"
INFO[2023-09-13 12:49:00.082754739Z] Checkpoint/restore support disabled           file="config/config.go:1134"
INFO[2023-09-13 12:49:00.082785402Z] Using seccomp default profile when unspecified: true  file="seccomp/seccomp.go:99"
INFO[2023-09-13 12:49:00.082814936Z] Using the internal default seccomp profile    file="seccomp/seccomp.go:152"
INFO[2023-09-13 12:49:00.082853209Z] AppArmor is disabled by the system or at CRI-O build-time  file="apparmor/apparmor_linux.go:34"
INFO[2023-09-13 12:49:00.082878960Z] No blockio config file specified, blockio not configured  file="blockio/blockio.go:37"
INFO[2023-09-13 12:49:00.082903192Z] RDT not available in the host system          file="rdt/rdt.go:56"
DEBU[2023-09-13 12:49:00.082944802Z] Using conmon from $PATH: /usr/bin/conmon      file="config/config.go:1379"
INFO[2023-09-13 12:49:00.083320448Z] Conmon does support the --sync option         file="conmonmgr/conmonmgr.go:85"
INFO[2023-09-13 12:49:00.083352048Z] Conmon does support the --log-global-size-max option  file="conmonmgr/conmonmgr.go:71"
INFO[2023-09-13 12:49:00.084596381Z] Found CNI network crio (type=bridge) at /etc/cni/net.d/11-crio-ipv4-bridge.conflist  file="ocicni/ocicni.go:343"
INFO[2023-09-13 12:49:00.084635727Z] Updated default CNI network name to crio      file="ocicni/ocicni.go:375"
DEBU[2023-09-13 12:49:00.084942414Z] reading hooks from /usr/share/containers/oci/hooks.d  file="hooks/read.go:65"
INFO[2023-09-13 12:49:00.085010492Z] Attempting to restore irqbalance config from /etc/sysconfig/orig_irq_banned_cpus  file="server/server.go:430"
INFO[2023-09-13 12:49:00.085064472Z] Restore irqbalance config: failed to get current CPU ban list, ignoring  file="runtimehandlerhooks/high_performance_hooks.go:565"
DEBU[2023-09-13 12:49:00.086620257Z] Golang's threads limit set to 460710          file="server/server.go:380"
WARN[2023-09-13 12:49:00.086758945Z] Error encountered when checking whether cri-o should wipe containers: open /var/run/crio/version: no such file or directory  file="server/server.go:640"
DEBU[2023-09-13 12:49:00.086890395Z] Sandboxes: []                                 file="server/server.go:545"
INFO[2023-09-13 12:49:00.086951885Z] Registered SIGHUP reload watcher              file="server/server.go:597"
DEBU[2023-09-13 12:49:00.087026006Z] Metrics are disabled                          file="server/server.go:555"
INFO[2023-09-13 12:49:00.087065645Z] Starting seccomp notifier watcher             file="server/server.go:758"
INFO[2023-09-13 12:49:00.087211028Z] Create NRI interface                          file="nri/nri.go:94"
INFO[2023-09-13 12:49:00.087230592Z] NRI interface is disabled in the configuration.  file="nri/nri.go:101"
DEBU[2023-09-13 12:49:00.111900051Z] monitoring "/usr/share/containers/oci/hooks.d" for hooks  file="hooks/monitor.go:43"

@xmudrii everything looks good so far, we just need a new home project for the package (maybe a new one for v1.29.0 as well).

xmudrii · 2023-09-13T13:54:28Z

I'll help with a new home project for the package, let's sync about this tomorrow.

saschagrunert · 2023-09-19T10:29:37Z

I think this can be considered done now, we just have to publish a bunch of releases and slight upstream fixes like cri-o/cri-o#7312

afbjorklund · 2023-09-22T10:59:58Z

The packaging of "cri-o" probably needs something similar to the packaging of "containerd.io" :

Conflicts: conmon, crun
Replaces: conmon, crun

To help sort out the conflicts with the regular system packages, that also installs the same binaries.

/usr/bin/conmon
/usr/bin/crun

saschagrunert · 2023-09-22T11:04:48Z

Yeah I'm going with the runtime dependencies here: #3282

afbjorklund · 2023-09-22T11:06:06Z

Hard to win that game, since it will then get whatever versions of conmon and crun the system has...

crun/unknown,now 100:1.2-2 amd64 [installed,automatic]
conmon/unknown,now 100:2.1.2~0 amd64 [installed,automatic]

saschagrunert · 2023-09-22T11:08:52Z

It will ensure somehow compatibility with the podman package, not sure. We can either break this or that 🤔

saschagrunert added kind/feature Categorizes issue or PR as related to a new feature. sig/release Categorizes an issue or PR as relevant to SIG Release. area/release-eng Issues or PRs related to the Release Engineering subproject labels Aug 21, 2023

k8s-ci-robot added the needs-priority label Aug 21, 2023

saschagrunert added the priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. label Aug 21, 2023

k8s-ci-robot removed the needs-priority label Aug 21, 2023

saschagrunert mentioned this issue Aug 25, 2023

Integrate CRI-O into krel obs #3229

Merged

saschagrunert mentioned this issue Sep 6, 2023

Integrate packages into Kubernetes release project cri-o/cri-o#7280

Closed

This was referenced Sep 13, 2023

Add /var/lib/crio to specfile #3268

Merged

Require iptables and CNI plugins for CRI-O #3269

Merged

This was referenced Sep 15, 2023

Add OBS package build jobs cri-o/cri-o#7310

Closed

Add registries.conf to CRI-O spec #3275

Merged

Fix CRI-O unitdir macro on debbuild #3274

Merged

[1.28] Add registries.conf to CRI-O bundle cri-o/cri-o#7312

Merged

saschagrunert closed this as completed Sep 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate CRI-O into `krel obs` #3214

Integrate CRI-O into `krel obs` #3214

saschagrunert commented Aug 21, 2023 •

edited

Loading

haircommander commented Aug 22, 2023

saschagrunert commented Aug 23, 2023

saschagrunert commented Sep 13, 2023

saschagrunert commented Sep 13, 2023

xmudrii commented Sep 13, 2023

saschagrunert commented Sep 19, 2023

afbjorklund commented Sep 22, 2023 •

edited

Loading

saschagrunert commented Sep 22, 2023

afbjorklund commented Sep 22, 2023 •

edited

Loading

saschagrunert commented Sep 22, 2023

Integrate CRI-O into krel obs #3214

Integrate CRI-O into krel obs #3214

Comments

saschagrunert commented Aug 21, 2023 • edited Loading

Useful information

haircommander commented Aug 22, 2023

saschagrunert commented Aug 23, 2023

saschagrunert commented Sep 13, 2023

saschagrunert commented Sep 13, 2023

Test of the rpm

Test of the deb

xmudrii commented Sep 13, 2023

saschagrunert commented Sep 19, 2023

afbjorklund commented Sep 22, 2023 • edited Loading

saschagrunert commented Sep 22, 2023

afbjorklund commented Sep 22, 2023 • edited Loading

saschagrunert commented Sep 22, 2023

Integrate CRI-O into `krel obs` #3214

Integrate CRI-O into `krel obs` #3214

saschagrunert commented Aug 21, 2023 •

edited

Loading

Test of the `rpm`

Test of the `deb`

afbjorklund commented Sep 22, 2023 •

edited

Loading

afbjorklund commented Sep 22, 2023 •

edited

Loading