Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zfs-2.3.0-rc2 patchset #16630

Draft
wants to merge 20 commits into
base: zfs-2.3-release
Choose a base branch
from

Commits on Oct 9, 2024

  1. ZTS: Remove FreeBSD 13.4-STABLE

    Current CI is failing on FreeBSD 13.4-STABLE, because samba4 can't be
    installed there. Lets remove it for now.
    
    Update also the FreeBSD version definitions a bit.
    
    The naming is like this now:
    
    FreeBSD variants:
    - freebsd13-3r, freebsd13-4r, freebsd14-0r, freebsd14-1r (RELEASE)
    - freebsd13-4s, freebsd14-1s (STABLE)
    - freebsd15-0c (CURRENT)
    
    RHL based distros:
    - almalinux8, almalinux9, centos-stream9, fedora39, fedora40
    
    Debian based:
    - debian11, debian12, ubuntu20, ubuntu22, ubuntu24
    
    Misc Linux distros:
    - archlinux, tumbleweed
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Closes openzfs#16610
    mcmilk authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    03822a6 View commit details
    Browse the repository at this point in the history
  2. ZTS: Fix summary page creation again - second try

    In PR openzfs#16599 I used 'return' like in C - which is wrong :/
    This fix generates the summary as needed.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Closes openzfs#16611
    mcmilk authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    f019b44 View commit details
    Browse the repository at this point in the history
  3. Fix generation of kernel uevents for snapshot rename on linux

    `zvol_rename_minors()` needs to be given the full path not just the
    snapshot name.  Use code removed in a0bd735 as a guide
    to providing the necessary values.
    
    Add ZTS check for /dev changes after snapshot rename.  After
    renaming a snapshot with 'snapdev=visible' ensure that the /dev
    entries are updated to reflect the rename.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: James Dingwall <james@dingwall.me.uk>
    Closes openzfs#14223 
    Closes openzfs#16600
    JKDingwall authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    1ebb6b8 View commit details
    Browse the repository at this point in the history
  4. contrib/debian: add new manpages to installation list

    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
    Closes openzfs#16609
    Harry-Chen authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    d32c059 View commit details
    Browse the repository at this point in the history
  5. Bump SONAME of libzfs and libzpool

    The ABI of libzfs and libzpool have breaking changes since last
    SONAME bump in commit fe6babc:
    
    * libzfs: `zpool_print_unsup_feat` removed (used by zpool cmd).
    * libzpool: multiple `ddt_*` symbols removed (used by zdb cmd).
    
    Bump them to avoid ABI breakage.
    
    See: openzfs#11817
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
    Closes openzfs#16609
    Harry-Chen authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    fc9608e View commit details
    Browse the repository at this point in the history
  6. Return boolean_t in inline functions of lib/libspl/include/sys/uio.h

    The inline functions zfs_dio_offset_aligned(), zfs_dio_size_aligned()
    and zfs_dio_aligned() are declared as boolean_t but return the bool
    type.
    
    This fixes the build of FreeBSD.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Martin Matuska <mm@FreeBSD.org>
    Closes openzfs#16613
    mmatuska authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    84f44ec View commit details
    Browse the repository at this point in the history
  7. zpool/zfs: restore -V & --version options

    The -j option added a round of getopt, which didn't know the magic
    version flags. So just bypass the whole thing and go straight to the
    human output function for the special case.
    
    Sponsored-by: https://despairlabs.com/sponsor/
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Umer Saleem <usaleem@ixsystems.com>
    Signed-off-by: Rob Norris <robn@despairlabs.com>
    Closes openzfs#16615 
    Closes openzfs#16617
    robn authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    0fbe9d3 View commit details
    Browse the repository at this point in the history
  8. Only serialize native-deb* targets

    .NOTPARALLEL target is being forced on userspace as well. This commit
    removes .NOTPARALEL target and only serializes the execution of
    native-deb* targets.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Umer Saleem <usaleem@ixsystems.com>
    Closes openzfs#16622
    usaleem-ix authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    0df10dc View commit details
    Browse the repository at this point in the history
  9. ZTS: resilver_restart_001.ksh restore defaults

    Update resilver_restart_001.ksh to restore the default
    resilver_defer_percent when the test completes.
    
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: Pavel Snajdr <snajpa@snajpa.net>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes openzfs#16618
    behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    10f46d2 View commit details
    Browse the repository at this point in the history
  10. vdev_id: multi-lun disks & slot num zero pad

    Add ability to generate disk names that contain both a slot number
    and a lun number in order to support multi-actuator SAS hard drives
    with multiple luns. Also add the ability to zero pad slot numbers to
    a desired digit length for easier sorting.
    
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Matthew Heller <matthew.f.heller@accre.vanderbilt.edu>
    Closes openzfs#16603
    hellermf authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    2609d93 View commit details
    Browse the repository at this point in the history
  11. ztest: Fix scrub check in ztest_raidz_expand_check()

    The scrub code may return EBUSY under several possible scenarios
    causing ztest to incorrectly ASSERT when verifying the result of
    a raidz expansion.  Update the test case to allow EBUSY since it
    does not indicate pool damage.
    
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes openzfs#16627
    behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    09f6b2e View commit details
    Browse the repository at this point in the history
  12. FreeBSD: ignore some includes when not building kernel

    The function abd_alloc_from_pages() is used only in kernel.
    Excluding sys/vm.h, and vm/vm_page.h includes avoids dependency
    problems.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Martin Matuska <mm@FreeBSD.org>
    Closes openzfs#16616
    mmatuska authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    774dcba View commit details
    Browse the repository at this point in the history
  13. Always validate checksums for Direct I/O reads

    This fixes an oversight in the Direct I/O PR. There is nothing that
    stops a process from manipulating the contents of a buffer for a
    Direct I/O read while the I/O is in flight. This can lead checksum
    verify failures. However, the disk contents are still correct, and this
    would lead to false reporting of checksum validation failures.
    
    To remedy this, all Direct I/O reads that have a checksum verification
    failure are treated as suspicious. In the event a checksum validation
    failure occurs for a Direct I/O read, then the I/O request will be
    reissued though the ARC. This allows for actual validation to happen and
    removes any possibility of the buffer being manipulated after the I/O
    has been issued.
    
    Just as with Direct I/O write checksum validation failures, Direct I/O
    read checksum validation failures are reported though zpool status -d in
    the DIO column. Also the zevent has been updated to have both:
    1. dio_verify_wr -> Checksum verification failure for writes
    2. dio_verify_rd -> Checksum verification failure for reads.
    This allows for determining what I/O operation was the culprit for the
    checksum verification failure. All DIO errors are reported only on the
    top-level VDEV.
    
    Even though FreeBSD can write protect pages (stable pages) it still has
    the same issue as Linux with Direct I/O reads.
    
    This commit updates the following:
    1. Propogates checksum failures for reads all the way up to the
       top-level VDEV.
    2. Reports errors through zpool status -d as DIO.
    3. Has two zevents for checksum verify errors with Direct I/O. One for
       read and one for write.
    4. Updates FreeBSD ABD code to also check for ABD_FLAG_FROM_PAGES and
       handle ABD buffer contents validation the same as Linux.
    5. Updated manipulate_user_buffer.c to also manipulate a buffer while a
       Direct I/O read is taking place.
    6. Adds a new ZTS test case dio_read_verify that stress tests the new
       code.
    7. Updated man pages.
    8. Added an IMPLY statement to zio_checksum_verify() to make sure that
       Direct I/O reads are not issued as speculative.
    9. Removed self healing through mirror, raidz, and dRAID VDEVs for
       Direct I/O reads.
    
    This issue was first observed when installing a Windows 11 VM on a ZFS
    dataset with the dataset property direct set to always. The zpool
    devices would report checksum failures, but running a subsequent zpool
    scrub would not repair any data and report no errors.
    
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
    Closes openzfs#16598
    bwatkinson authored and behlendorf committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    26ecd8b View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. zpool/zfs: allow --json wherever -j is allowed

    Mostly so that with the JSON formatting options are also used, they all
    look the same. To my eye, `-j --json-flat-vdevs` suggests that they are
    different or unrelated, while `--json --json-flat-vdevs` invites no
    further questions.
    
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Reviewed-by: Umer Saleem <usaleem@ixsystems.com>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
    Closes openzfs#16632
    robn authored and behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    6669036 View commit details
    Browse the repository at this point in the history
  2. zdb: fix printf format in dump_zap()

    When compiling zdb.c on 32-bit platforms, a format conversion error 
    is reported for a printf() in dump_zap().  Change %l to macro 
    %" PRIu64 " to match the platform size of a 64-bit unsigned integer.
    
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Martin Matuska <mm@FreeBSD.org>
    Closes openzfs#16635
    mmatuska authored and behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5816296 View commit details
    Browse the repository at this point in the history
  3. CI: Stick with ubuntu-22.04 for CodeQL analysis

    The ubuntu-latest alias now refers to ubuntu-24.04 instead of
    ubuntu-22.04 which causes CodeQL's autobuild to fail with:
    
        cpp/autobuilder: deptrace not supported in ubuntu 24.04
    
    Until deptrace is supported by ubuntu-24.04 hosted runners request
    ubuntu-22.04 which is supported.
    
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Closes openzfs#16639
    behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    7f830d7 View commit details
    Browse the repository at this point in the history
  4. ZTS: Slightly increase dedup_quota limit

    As described in the comment above this check the space used by
    logged entries is not accounted for and some margin needs to be
    added in.  While uncommon we have slightly exceeded the 600,000
    threshold on some CI run so we increase the limit a bit more.
    
    Reviewed-by: Alexander Motin <mav@FreeBSD.org>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes openzfs#16637
    behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5bc27ac View commit details
    Browse the repository at this point in the history
  5. ZTS: Increase zpool_import_parallel_pos import margin

    Increase the pool import time allowed by assuming a minimum reduction
    to 1/2 instead of 1/3 when comparing sequential to parallel import
    times.  This is sufficient to verify parallel imports are working as
    intended and should address the occasional false positive failure
    when the time is slightly exceeded.
    
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Reviewed-by: George Melikov <mail@gmelikov.ru>
    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes openzfs#16638
    behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    c645b07 View commit details
    Browse the repository at this point in the history
  6. Fallback to strerror() when strerror_l() isn't available

    Some C libraries, such as uClibc, do not provide strerror_l() in
    which case we fallback to strerror().
    
    Reviewed-by: Tino Reichardt <milky-zfs@mcmilk.de>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    85efa64 View commit details
    Browse the repository at this point in the history
  7. Tag 2.3.0-rc2

    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    behlendorf committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    12c5740 View commit details
    Browse the repository at this point in the history