v34

• Improvements

  • Drop pre-built .ko modules from git - distros/packages will need the
    linux-headers to be able to run the testsuite. There was limited use
    of the feature, while linters complained about "source-not-included"
    or "source-contains-prebuilt-binary".

  • Switch build system to meson: autotools is still supported but slated
    for removal on next release. This is the transition release to help
    distros and integrators to move to the new build system. Default options
    target distros while developers can use the build-dev.ini configuration
    file.

  • Allow to load decompression libraries ondemand: liblzma.so, libz.so,
    libxz.so and libzstd.so can now be loaded ondemand, only when there is
    such a need. For use during early boot for loading modules, if
    configured well it means none of these libraries are loaded: the
    module loading logic via finit_module() will just hand over to kernel
    the open file descriptor and kernel will use its own decompress routine.

    If kernel doesn't handle decompression or if the module is compressed
    with a different algorithm than the one configured in the kernel,
    libkmod can still open the module by dynamically loading the
    correspondent library.

    Tools inspecting the module contents, like modinfo, will load that
    single decompression library instead all of them.

    For distros building with meson it's possible to choose the behavior
    per library. Examples: a) -Ddlopen=all uses dlopen behavior for all
    the libraries; b) -Ddlopen=xz, will make only xz to be dlopen'ed
    while other enabled libraries will be linked in at build time.

    The use of dlopen is annotated in the ELF file by using the ELF
    Package Metadata spec as documented in
    https://systemd.io/ELF_PACKAGE_METADATA/. Example:

    $ dlopen-notes.py libkmod.so
    # build/libkmod.so
    [
      {
        "feature": "xz",
        "description": "Support for uncompressing xz-compressed modules",
        "priority": "recommended",
        "soname": [
          "liblzma.so.5"
        ]
      }
    ]
    

  • Add -m / --moduledir to depmod to override in runtime the module
    directory that was already possible to set on build time. Document
    the interaction between the dir options: base, module and output.

  • Better error propagation in libkmod for its internal APIs and libc
    functions up to the callers.

  • Improve libkmod API documentation by adding new sections, documenting
    functions previously missing, rewording existing ones, adding version
    information, cross-referencing, etc.

  • Remove deprecated arguments for depmod: --unresolved-error, --quiet,
    --root and --map.

  • Remove deprecated arguments for rmmod: -w.

  • Remove deprecated arguments for insmod: -p and -s.

  • Add --syslog and --force for insmod to normalize it with other tools.

  • Add bash, fish and zsh shell-completion for insmod, rmmod and lsmod.

  • Remove depmod_module_directory_override from .pc as the kernel side
    is not making use of it and will likely not need it.

  • Improve builtin module listing and retrieving information from its
    modinfo index which reduces the amount of needed syscalls by 90%.

  • Improve zstd decompression by using streaming bufferless mode which
    reduces the amount of syscalls by 65%.

  • Increase use of pread while parsing ELF and indexes in order to reduce
    syscalls and improve performance.

  • Improve module sorting in depmod to speedup the use of the
    modules.order index and support duplicate lines in it.

  • Avoid misaligned memory access while reading module signature in
    libkmod.

  • Add more documentation for contributing to kmod. New developers are
    welcome to look at the new README.md and CONTRIBUTING.md files for
    information on process, coding style, build/installation, etc.

  • Overhaul man pages with multiple clarifications, section rewrites and
    additional documentation.

  • Drop --with-rootlibdir as it's seldom used and was partially broken.

  • Drop strndupa() and alloca() for increased libc compatibility.

  • Better handling of LFS for increased compatibility with libc.

  • Protect kmod_get_dirname() and kmod_new() against NULL argument.

  • Normalize --version / --help output across all tools.

  • Always include log priority in messages, even when building with debug.

  • Optimize index reading by lazily reading nodes on demand, reducing
    FILE overhead and reducing code duplication wrt FILE vs mmap
    implementations, etc.

  • Switch index to pre-order to improve performance in both read and
    write, meaning faster lookup and faster depmod. Some examples:
    a) traversing all indexes via configuration dump shows a 9%
    improvement on Raspberry Pi 2. b) writing the indexes takes 90% less
    lseek() calls, leading to a performance gain of 13%.

  • Make symlink install locations more similar to what distros are
    using: by default it installs the kmod binary as bin/kmod and the
    symlinks are located in e.g. sbin/depmod -> ../bin/kmod. Changing
    the sbin location is sufficient to move the symlinks to the
    appropriate place, so distros using --sbin /usr/bin will have them
    installed in that directory. This avoids distros having to remove the
    symlink and add the symlinks by themselves. (meson only)

  • Install configuration directories,
    /{etc,usr/lib}/{depmod,modprobe}.d/ as part of installation, matching
    what several distros do during packaging. (mson only)

• Bug fixes

  • Fix testsuite using when using configurable module dir.

  • Fix typos on documentation and source code.

  • Fix out of bound access in multiple places when using long paths,
    synthetic huge files, or handling memory allocation errors, or
    inconsistent variable types, particularly on 32b builds.

  • Fix internal array APIs, with better error checking: improve execution on
    very memory-constrained scenarios or very long paths.

  • Fix absolute path handling in depmod.

  • Fix libkmod memory leaks on error handling when getting builtin
    module list.

  • Do not crash on invalid modules.builtin.modinfo file.

  • Fix link with lld resulting in empty testsuite.

  • Fix testsuite build/execution with musl.

• Others

  • Adopt clang-format and editorconfig for coding style and setup CI
    action to keep the codebase consistent.

  • Adopt codespell in CI.

  • Adopt CodeQL integration in CI.

  • Adopt Codecov in CI.

  • Adopt SPDX copyright and license identifiers throughout the project.

  • Add more distros to CI, 32b builds, clang as compiler and lld as
    linker.

  • Add support for clang sanitizers and squelch warnings.

  • Add tests for builtin modules from modinfo index file.

  • Multiple testsuite refactors and fixes to make it simpler to write tests.

  • Add CI coverage for docs

  • Improve strbuf implementation with more error checks and generalize
    it to cover the role of scratchbuf. This allows to remove the
    scratchbuf implementation.

  • Use common array and strbuf code in depmod to remove duplication.

  • Add abstraction and use more compiler builtins for addition and
    multiplication with overflow checking.

  • Normalize use of C attributes throughout the project.

kmod 30

• Improvements

  • Stop adding duplicate information on modules.builtin.alias.bin, just use
    the modules.builtin.bin index

  • Speedup depmod, particularly under qemu with emulated arch, by
    avoiding a lot of open/read/close of modules.alias.bin. On an
    emulated ARM rootfs, depmod with only 2 modules was taking ~32s
    vs ~0.07s now.

  • Add kmod_module_new_from_name_lookup() which allows doing a lookup by
    module name, without considering the aliases. Other than that search
    order is similar to kmod_module_new_from_lookup().

  • modinfo learned the --modname option to explicitely show information
    about the module, even if there is an alias with the same name. This
    allows showing information about e.g. kernel/lib/crc32.ko, even if
    kernel also exports a crc32 alias in modules.alias:

            alias crc32 crc32_pclmul
            alias crc32 crc32_generic
    
      Same behavior will be used to other modules and to aliases provided
      by user/distro.
    

  • depmod.conf learned a new "excludedir" directive so distro/user can
    configure more directories to be excluded from its search, besides
    the hardcoded values "build" and "source".

  • Better group modprobe options on help output under "Management, Query and General".

  • modprobe learned a --wait option to be used together with -r
    when removing a module. This allows modprobe to keep trying the
    removal if it fails because the module is still in use. An exponential backoff
    time is used for further retries.
    The wait behavior provided by the kernel when not passing O_NONBLOCK
    to delete_module() was removed in v3.13 due to not be used and the
    consequences of having to support it in the kernel. However there may
    be some users, particularly on testsuites for individual susbsystems, that
    would want that. So provide a userspace implementation inside modprobe for
    such users. "rmmod" doesn't have a --wait as it remains a bare minimal over
    the API provided by the kernel. In future the --wait behavior can be added
    to libkmod for testsuites not exec'ing modprobe for module removal.

  • kmod_module_remove_module() learned a new flag to silence output when
    caller wants to handle them - this is particularly important for the
    --wait flag to modprobe, as it's not desired to keep seeing error messages
    while waiting for the module to be unused.

  • Add SM3 hash algo support to modinfo output, as already available in the kernel.

• Bug Fixes

  • Fix modinfo output when showing information for a .ko module when running
    on a kernel that has that module as builtin.

  • Fix kmod_module_new_from_lookup() returning > 0 rather than 0
    when it matches an alias.

  • Fix modinfo segfault when module doesn't exist.

  • Add missing function in the html documentation: kmod_get_dirname().

  • Fix modprobe incorrectly handling number of arguments when prepending values from
    MODPROBE_OPTIONS environment variable.

  • Fix modprobe -r --remove-dependencies and since "dependencies" was a
    misnomer, add the preferred argument option: "--remove-holders". This
    is the same name used by the kernel. It allows users to also remove
    other modules holding the one that is being removed.

  • Fix off-by-one in max module name length in depmod.

• Infra/internal

  • Start some changes in the out-of-tree test modules in kmod so they are useful
    for being really inserted in the kernel rather than relying on kmod's mock
    interface. This helps manual testing and may be used to exercise to test
    changes in the kernel.

kmod 29

• Improvements

  • Add support to use /usr/local as a place for configuration files. This makes it easier
    to install locally without overriding distro files.

• Bug fixes

  • Fix modinfo -F when module is builtin: when we asked by a specific field from modinfo,
    it was not working correctly if the module was builtin
  • Documentation fixes on precedence order of /etc and /run: the correct order is
    /etc/modprobe.d, /run/modprobe.d, /lib/modprobe.d
  • Fix the priority order that we use for searching configuration files. The
    correct one is /etc, /run, /usr/local/lib, /lib, for both modprobe.d
    and depmo.d
  • Fix kernel command line parsing when there are quotes present. Grub
    mangles the command line and changes it from 'module.option="val with
    spaces"' to '"module.option=val with spaces"'. Although this is weird
    behavior and grub could have been fixed, the kernel understands it
    correctly for builtin modules. So change libkmod to also parse it
    correctly. This also brings another hidden behavior from the kernel:
    newline in the kernel command line is also allowed and can be used to
    separate options.
  • Fix a memory leak, overflow and double free on error path
  • Fix documentation for return value from kmod_module_get_info(): we
    return the number of entries we added to the list
  • Fix output of modules.builtin.alias.bin index: we were writing an empty file due to
    the misuse of kmod_module_get_info()

• Infra/internal

  • Retire integration with semaphoreci
  • Declare the github mirror also as an official upstream source: now besides accepting
    patches via mailing list, PRs on github are also acceptable
  • Misc improvements to testsuite, so we can use it reliably regardless
    of the configuration used: now tests will skip if we don't have the
    build dependencies)