This document explains how to modify & update initrd for Ubuntu Core
Sometimes you may need to alter the behaviour of stock initrd or you may want to add a new functionality to it.
Modifying initrd is relatively easy for arm SBC's (Like RPi) or amd64 devices ( Like Intel NUCs ) without secure boot ( No TPM or PTT or similar)
For devices with secure boot, one need to :
TODO: Explain how to modify initrd when there is Secure boot / FDE.
- A Linux Computer with various development tools
- Latest Ubuntu Desktop is recommended
- (For a SBC like RPi) you will need UART TTL serial debug cable (Because you will not have SSH in initrd)
- initramfs-tools
- snapcraft
- etc
You need to have the following software installed before you can test with spread
- Go (https://golang.org/doc/install or
sudo snap install go) - Spread (install from source as per below)
You can install spread by simply using snap install spread, however this does not allow for the lxd-backend to be used.
To use the lxd backend you need to install spread from source, as the LXD profile support has not been upstreamed yet.
This document will be updated with the upstream version when the PR canonical/spread#136 merges. To install spread from source you need to do the following.
git clone https://github.com/Meulengracht/spread
cd spread
cd cmd/spread
go build .
go install .
- Install the dependencies required for the qemu emulation
sudo apt update && sudo apt install -y qemu-kvm autopkgtest
- Create a suitable ubuntu test image (jammy) in the following directory where spread locates images. Note that the location is different when using spread installed through snap.
mkdir -p ~/.spread/qemu # This location is different if you installed spread from snap
cd ~/.spread/qemu
autopkgtest-buildvm-ubuntu-cloud -r jammy
- Rename the newly built image as the name will not match what spread is expecting
mv autopkgtest-jammy-amd64.img ubuntu-22.04-64.img
- Now you are ready to run spread tests with the qemu backend
cd ~/core-initrd # or wherever you checked out this repository
spread qemu-nested
The LXD backend is the preffered way of testing locally as it uses virtualization and thus runs a lot quicker than the qemu backend. This is because the container can use all the resources of the host, and we can support qemu-kvm acceleration in the container for the nested instance.
This backend requires that your host machine supports KVM.
- Setup any prerequisites and build the LXD image needed for testing. The following commands will install lxd and yq (needed for yaml manipulation), download the newest image and import it into LXD.
sudo snap install lxd
sudo snap install yq
curl -o lxd-initrd-img.tar.gz https://storage.googleapis.com/snapd-spread-core/lxd/lxd-spread-initrd22-img.tar.gz
lxc image import lxd-initrd-img.tar.gz --alias ucspread22
lxc image show ucspread22 > temp.profile
yq e '.properties.aliases = "ucspread22,amd64"' -i ./temp.profile
yq e '.properties.remote = "images"' -i ./temp.profile
cat ./temp.profile | lxc image edit ucspread22
rm ./temp.profile ./lxd-initrd-img.tar.gz
- Import the LXD coreinitrd test profile. Make sure your working directory is the root of this repository.
lxc profile create coreinitrd
cat tests/spread/core-initrd.lxdprofile | lxc profile edit coreinitrd
- Set environment variable to enable KVM acceleration for the nested qemu instance
export SPREAD_ENABLE_KVM=true
- Now you can run the spread tests using the LXD backend
spread lxd-nested
Getting a debug shell in initrd is very simple:
- Boot your UC22 image on your RPi
- Access to it via SSH
- Edit your kernel commandline:
First your current kernel commandline is:
~$ cat /run/mnt/ubuntu-seed/cmdline.txt
dwc_otg.lpm_enable=0 console=serial0,115200 elevator=deadline rng_core.default_quality=700 vt.handoff=2 quiet splash
~$
Now you need to add following three arguments to it:
rd.systemd.debug_shell=serial0 dangerous rd.systemd.unit=emergency.service
So it will look like:
~$ cat /run/mnt/ubuntu-seed/cmdline.txt
dwc_otg.lpm_enable=0 console=serial0,115200 elevator=deadline rng_core.default_quality=700 vt.handoff=2 quiet splash rd.systemd.debug_shell=serial0 dangerous rd.systemd.unit=emergency.service
~$
Warning: Because of a bug in boot.scr, following will not happen, and your RPi will boot to main rootfs again (you can ssh to it). Leaving this warning until this bug is fixed.
Finally, after a reboot, you will get a shell like this:
PM_RSTS: 0x00001000
....lot of logs here
Starting start4.elf @ 0xfec00200 partition 0
U-Boot 2020.10+dfsg-1ubuntu0~20.04.2 (Jan 08 2021 - 13:03:11 +0000)
DRAM: 3.9 GiB
....lot of logs
Decompressing kernel...
Uncompressed size: 23843328 = 0x16BD200
20600685 bytes read in 1504 ms (13.1 MiB/s)
Booting Ubuntu (with booti) from mmc 0:...
## Flattened Device Tree blob at 02600000
Booting using the fdt blob at 0x2600000
Using Device Tree in place at 0000000002600000, end 000000000260f07f
Starting kernel ...
[ 1.224420] spi-bcm2835 fe204000.spi: could not get clk: -517
BusyBox v1.30.1 (Ubuntu 1:1.30.1-4ubuntu6.3) built-in shell (ash)
Enter 'help' for a list of built-in commands.
#
In this state, if you want to pivot-root to main rootfs, just execute:
# systemctl start basic.target
Sometimes, in order for testing some new feature, rebuilding is not necessary. It is possible to add your fancy nice application to current initrd.img. Depending on the board, we need to do different things, so we will show how to do this for an RPi and for x86 images.
Basically:
- Download current initrd.img from your RPi to your host
- unpack it to a directory
- add your binary / systemd unit into it
- repack initrd.img
- upload to your RPi
- reboot
~ $ mkdir uc-initrd
~ $ cd uc-initrd/
~/uc-initrd $ scp pi:/run/mnt/ubuntu-boot/uboot/ubuntu/pi-kernel_292.snap/initrd.img .
initrd.img 100% 20MB 34.8MB/s 00:00
~/uc-initrd $ unmkinitramfs initrd.img somedir
~/uc-initrd $ tree -L 1 somedir/
somedir/
├── bin -> usr/bin
├── etc
├── init -> usr/lib/systemd/systemd
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── sbin -> usr/bin
└── usr
5 directories, 2 files
~/uc-initrd $ echo "echo \"Hello world\"" > somedir/usr/bin/hello.sh
~/uc-initrd $ chmod +x somedir/usr/bin/hello.sh
~/uc-initrd $ somedir/usr/bin/hello.sh
Hello world
~/uc-initrd $
~/uc-initrd $ cd somedir/
~/uc-initrd/somedir $ find ./ | cpio --create --quiet --format=newc --owner=0:0 | lz4 -9 -l > ../initrd.img.new
103890 blocks
~/uc-initrd/somedir $ cd ..
~/uc-initrd $ file initrd.img
initrd.img: LZ4 compressed data (v0.1-v0.9)
~/uc-initrd $ file initrd.img.new
initrd.img.new: LZ4 compressed data (v0.1-v0.9)
~/uc-initrd $ ll
total 40252
drwxrwxr-x 3 4096 Haz 21 17:53 ./
drwxr-x--- 36 4096 Haz 21 17:52 ../
-rwxr-xr-x 1 20600685 Haz 21 17:43 initrd.img*
-rw-rw-r-- 1 20601051 Haz 21 17:53 initrd.img.new
drwxrwxr-x 4 4096 Haz 21 17:49 somedir/
~/uc-initrd $ scp initrd.img.new pi:/tmp/
initrd.img.new 100% 20MB 41.6MB/s 00:00
~/uc-initrd $ ssh pi
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-1037-raspi aarch64)
......
Last login: Mon Jun 21 14:40:10 2021 from 192.168.1.247
~$ sudo cp /tmp/initrd.img.new /run/mnt/ubuntu-boot/uboot/ubuntu/pi-kernel_292.snap/initrd.img
~$ sync
~$ sudo reboot
Connection to 192.168.1.224 closed by remote host.
Connection to 192.168.1.224 closed.
~/uc-initrd $
And then, finally, on your serial console :
Starting kernel ...
[ 1.223730] spi-bcm2835 fe204000.spi: could not get clk: -517
BusyBox v1.30.1 (Ubuntu 1:1.30.1-4ubuntu6.3) built-in shell (ash)
Enter 'help' for a list of built-in commands.
# hello.sh
Hello world
#
For x86 the process is a bit different as in this case the initrd is a section in a PE+/COFF binary. We can download the snap and extract the initrd with:
$ snap download --channel=20/stable pc-kernel
$ unsquashfs -d pc-kernel pc-kernel_*.snap
$ objcopy -O binary -j .initrd pc-kernel/kernel.efi initrd.img
$ unmkinitramfs initrd.img initrd
Then, you can change the initrd as described in the RPi section. After that, to repack everything, run these commands:
$ cd initrd
$ find . | cpio --create --quiet --format=newc --owner=0:0 | lz4 -l -7 > ../initrd.img
$ cd -
$ sudo add-apt-repository ppa:snappy-dev/image
$ apt download ubuntu-core-initramfs
$ dpkg --fsys-tarfile ubuntu-core-initramfs_*_amd64.deb |
tar xf - ./usr/lib/ubuntu-core-initramfs/efi/linuxx64.efi.stub
$ objcopy -O binary -j .linux pc-kernel/kernel.efi linux
$ objcopy --add-section .linux=linux --change-section-vma .linux=0x2000000 \
--add-section .initrd=initrd.img --change-section-vma .initrd=0x3000000 \
usr/lib/ubuntu-core-initramfs/efi/linuxx64.efi.stub \
pc-kernel/kernel.efi
$ snap pack pc-kernel
You can use this new kernel snap while building image, or copy it over
to your device and install. Note that the new kernel.efi won't be
signed, so Secure Boot will not be possible anymore.
The initrd is part of the kernel snap, so ideally we would prefer to simply build it by using snapcraft. However, the snapcraft recipe for the kernel downloads binaries from already built kernel packages, so we cannot use it easily for local hacking. So we will provide instructions on how to build the initrd from scratch and insert it in the kernel snap.
First, we need to build the debian package, for this you can use debuild or dpkg-buildpackage from the root folder:
$ sudo apt build-dep ./
$ debuild -us -uc
Then, install the package in the container:
$ sudo apt install ../ubuntu-core-initramfs_*.deb
We extract the kernel from the pc-kernel snap:
$ snap download --channel=20/stable pc-kernel
$ unsquashfs -d pc-kernel pc-kernel_*.snap
We can now extract the kernel image from kernel.efi:
$ snap info pc-kernel | grep 20/stable
20/stable: 5.4.0-87.98.1 2021-09-30 (838) 295MB -
$ kernelver=5.4.0-87-generic
$ objcopy -O binary -j .linux pc-kernel/kernel.efi kernel.img-"${kernelver}"
We can inject it in the kernel snap as in previous sections. It is
also possible to force the build of kernel.efi (or just the initrd), with:
$ ubuntu-core-initramfs create-initrd --kernelver=$kernelver --kerneldir pc-kernel/modules/${kernelver} --firmwaredir pc-kernel/firmware --output initrd.img
$ ubuntu-core-initramfs create-efi --kernelver=$kernelver --initrd initrd.img --kernel kernel.img --output kernel.efi
Note that for RPi we need only the initrd image file.
Now kernel.efi-5.4.0-87-generic has been created with the new initramfs. We can put it back into the snap:
$ cp kernel.efi-$kernelver pc-kernel/kernel.efi
$ snap pack pc-kernel