-
Notifications
You must be signed in to change notification settings - Fork 103
Testing
Ktest is just a set of C headers with very close hierarchy to the original Linux, but much simplified and combined with some additional helpers. This headers provide only most usable stubs for real kernel headers to launch a relatively standalone logic under the test.
The good examples of the code suitable for the testing with the framework are data strucures like HTrie, mathematic functions like APM maths or multi-precission math.
The framework emphasises simplicity to make debugging of complicated and standalone logic as productive as possible: (re-)run quickly, easily traced with gdb and valgrind.
KUnit, available in the mainstream
kernels at linux/tools/testing/kunit/, is essentially a bunch of Python
scripts around User-mode Linux (UML).
UML is also in the mainstream kernel and one just needs to
buid the kernel for um
architecture.
The most important thing bout KUnit/UML is that this is a fully functional Linux kernel, so you need a fake filesystem, login to the console and so on. It can be debugged with gdb (see example), but special steps are required. All in all, this can be and should be treated as a virtualization solution.
Tests are developed for KUnit/UML as Linux kernel modules (this
LWN article provides a good intro into the
framework), which is somewhat close to current tempesta_fw/t/unit framework
operation. As discussed in another
LWN article the framework emulates device
initialization, making tests (re-)run faster than in KVM, which we use.
Ext4 tests, as one of the good use case for the framework, require a filesystem
anyhow and they do required significant underlaying layers (block IO) emulation.
This is example of non-standalone logic, which can not be tested with our
ktest framework.
Thus, KUnit/UML is closer to tempesta_fw/t/unit tests than to ktest. In
future, if tempesta_fw/t/unit grows or become more complex, e.g. require
the TCP stack, it could make sense to port tempesta_fw/t/unit to KUnit/UML.
Linux Kernel Library (see also LWN anritcle) build the Linux kernel as a library, so a user space application can use the Linux API. Unfortunately, the project isn't mainstream and it looks like an overkill for us to apply and support the Tempesta patch for the LKL kernel and build the separate kernel just for the couple tests for the standalone logic.
Debugging some parts of code requires setups which contain more than one NUMA node. Here is an instruction on how to create a VM with NUMA support.
sudo apt install qemu-kvm
sudo apt install cloud-image-utils
wget https://cloud-images.ubuntu.com/releases/jammy/release/ubuntu-22.04-server-cloudimg-amd64.img
qemu-img resize ubuntu-22.04-server-cloudimg-amd64.img +40G
Standard cloud image is user agnostic. To save credentials and other user information we need to create yet another disk image and connect it to our VM.
$ cat > user-data <<EOF
#cloud-config
password: <password>
chpasswd: { expire: False }
ssh_pwauth: True
EOF
cloud-localds seed.img user-data
qemu-system-x86_64 \
-enable-kvm \
-cpu host \
-m 8G \
-drive if=virtio,format=qcow2,file=ubuntu-22.04-server-cloudimg-amd64.img \
-drive if=virtio,format=raw,file=seed.img \
-smp 16,sockets=4,cores=4,threads=1,maxcpus=16 \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-object memory-backend-ram,size=2G,id=m2 \
-object memory-backend-ram,size=2G,id=m3 \
-numa node,nodeid=0,memdev=m0,cpus=0-3 \
-numa node,nodeid=1,memdev=m1,cpus=4-7 \
-numa node,nodeid=2,memdev=m2,cpus=8-11 \
-numa node,nodeid=3,memdev=m3,cpus=12-15 \
-device virtio-net-pci,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::2222-:22 \
-nographic
Login: ubuntu
Password: <password from the user-data file>
ssh -p 2222 ubuntu@localhost
On the VM run:
sudo apt install numactl
numactl -H
Should see NUMA configuration: 4 nodes with 4 CPUs and 2G memory as for configuration described above.
Inside created VM install build tools, download sources and patches.
Perform standard build commands.
make -j $(nproc)
make install
Then copy bzImage and initrd.img to the host.
On the host run:
scp -P 2222 ubuntu@localhost:linux-<version>/arch/x86/boot/bzImage .
scp -P 2222 ubuntu@localhost:/boot/initrd.img-<version> .
qemu-system-x86_64 \
-enable-kvm \
-cpu host \
-m 8G \
-drive if=virtio,format=qcow2,file=ubuntu-22.04-server-cloudimg-amd64.img \
-drive if=virtio,format=raw,file=seed.img \
-smp 16,sockets=4,cores=4,threads=1,maxcpus=16 \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-object memory-backend-ram,size=2G,id=m2 \
-object memory-backend-ram,size=2G,id=m3 \
-numa node,nodeid=0,memdev=m0,cpus=0-3 \
-numa node,nodeid=1,memdev=m1,cpus=4-7 \
-numa node,nodeid=2,memdev=m2,cpus=8-11 \
-numa node,nodeid=3,memdev=m3,cpus=12-15 \
-device virtio-net-pci,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::2222-:22 \
-kernel bzImage \
-initrd initrd.img-<version>
-nographic \
-append "root=/dev/vda1 console=ttyS0" \
-serial mon:stdio
At the moment we have only 3 types of tests and they are well covered by the current frameworks:
-
The most developed and the largest framework is the functional test Python framework (https://github.com/tempesta-tech/tempesta-test/). This type of the tests are most beneficial because (1) they emulate cases very close to the real life scenatios and (2) they test quite a bunch of logic at once.
-
Complex standalone logic (HTrie, TLS, math in
tempesta_fw/t/unit/user_space/) are covered with thektest. -
Linux kernel module for unit tests (https://github.com/tempesta-tech/tempesta/tree/master/tempesta_fw/t). At the moment the module is quite small and some of the tests, e.g. HTTP parser or TfwStr, are likely to be moved to ktest framework. However, at some point we might need to develop tests which (1) require to reproduce specific and rare scenarios unreachable for the functional tests framework and (2) depend on significant Linux kernel logic unmockable by the
ktest. At this moment we might need to use either LKL or KUnit/UML.