fixes to make zfs test suite happier #4

avg-I · 2016-03-22T17:02:58Z

No description provided.

... despite its name.

We have to account for that.

E.g. if we try to promote a filesystem that is not a clone.

dmu_objset_find() expects a name that is a filesystem or volume name. It gets confused if given a snapshot name.

First, this restores compatibility with historic behavior. Second, this makes sense, because it is okay for datasets to get removed between when we decide to iterate them and when we actually attempt that. In that case we behave as if the datasets didn't exist in the first place. This is especially useful when walking over complex dataset networks including children and clones (that can also be children).

Also, do not artificially set ZFS_ITER_DEPTH_LIMIT flag if -d option is not passed. zfs_for_each() already does the right thing without that flag set.

Previously the former was done even if the latter succeeded.

EDOM may occur if a user tries to set `recordsize` too large without use "zfs set". This can be demonstrated with: > zpool create testpool -O recordsize=32M /dev/... Signed-off-by: DHE <git@dehacked.net> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#3911 (cherry picked from commit 385f969)

5767 fix several problems with zfs test suite Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Christopher Siden <christopher.siden@delphix.com> Approved by: Gordon Ross <gwr@nexenta.com> References: https://www.illumos.org/issues/5767 illumos/illumos-gate@52244c0 Porting Notes: - Only the updates to zpool_main.c were kept because the ZFS test suite is not currently part of the ZoL source tree. The test suite itself should be updated to include the latest versions of the tests once we're running it for every commit - Fixes `zpool list` output. Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> (cherry picked from commit 8e4c5c9)

ryao · 2016-03-25T04:20:01Z

This looks good to me. I am merging it.

[257324.124447] VERIFY(!rrw_held(&dp->dp_config_rwlock, RW_READER)) failed [257324.124450] PANIC at dsl_pool.c:1052:dsl_pool_config_enter() [257324.124451] Showing stack for process 29206 [257324.124453] CPU: 6 PID: 29206 Comm: zfs Tainted: P OE 4.1.3 #4 [257324.124454] Hardware name: Supermicro X10SAE/X10SAE, BIOS 3.0 05/20/2015 [257324.124455] 000000000000041c ffff88040c7aba88 ffffffff8161db7f 0000000080000000 [257324.124457] ffffffffc0bf6238 ffff88040c7aba98 ffffffffc07f95c4 ffff88040c7abc28 [257324.124458] ffffffffc07f968b ffff88040c7abb58 ffff8807e82f34e8 ffff880400000028 [257324.124459] Call Trace: [257324.124465] [<ffffffff8161db7f>] dump_stack+0x4f/0x7b [257324.124484] [<ffffffffc07f95c4>] spl_dumpstack+0x44/0x50 [spl] [257324.124488] [<ffffffffc07f968b>] spl_panic+0xbb/0x100 [spl] [257324.124491] [<ffffffff8108547d>] ? preempt_count_add+0x5d/0xb0 [257324.124492] [<ffffffff810853e1>] ? get_parent_ip+0x11/0x50 [257324.124493] [<ffffffff8108547d>] ? preempt_count_add+0x5d/0xb0 [257324.124495] [<ffffffff816231cb>] ? _raw_spin_lock+0x1b/0x70 [257324.124496] [<ffffffff81623237>] ? _raw_spin_unlock+0x17/0x30 [257324.124501] [<ffffffffc07fd337>] ? tsd_hash_search.isra.0+0x77/0xa0 [spl] [257324.124502] [<ffffffff810853e1>] ? get_parent_ip+0x11/0x50 [257324.124503] [<ffffffff8108547d>] ? preempt_count_add+0x5d/0xb0 [257324.124504] [<ffffffff816231cb>] ? _raw_spin_lock+0x1b/0x70 [257324.124505] [<ffffffff81623237>] ? _raw_spin_unlock+0x17/0x30 [257324.124525] [<ffffffffc0b0e5d8>] dsl_pool_config_enter+0x68/0x70 [zfs] [257324.124539] [<ffffffffc0b10612>] dsl_pool_hold+0x42/0x50 [zfs] [257324.124550] [<ffffffffc0adfab6>] dmu_objset_hold+0x26/0xb0 [zfs] [257324.124571] [<ffffffffc0b7dca9>] zfs_ioc_objset_stats_impl.constprop.31+0x29/0x60 [zfs] [257324.124589] [<ffffffffc0b7e70d>] ? dataset_name_hidden+0x3d/0x50 [zfs] [257324.124607] [<ffffffffc0b7ec2b>] zfs_ioc_dataset_list_next_impl+0x12b/0x170 [zfs] [257324.124626] [<ffffffffc0b7ecf9>] zfs_ioc_list_next+0x89/0x280 [zfs] [257324.124629] [<ffffffffc07f1cec>] ? strdup+0x3c/0x70 [spl] [257324.124646] [<ffffffffc0b7ef20>] zfs_ioc_dataset_list_next+0x10/0x20 [zfs] [257324.124664] [<ffffffffc0b81b76>] zfsdev_ioctl+0x4e6/0x5b0 [zfs] [257324.124666] [<ffffffff811c2b90>] do_vfs_ioctl+0x2e0/0x4e0 [257324.124668] [<ffffffff81253757>] ? file_has_perm+0x87/0xa0 [257324.124669] [<ffffffff811c2e11>] SyS_ioctl+0x81/0xa0 [257324.124671] [<ffffffff81623c5b>] system_call_fastpath+0x16/0x6e Signed-off-by: Richard Yao <ryao@gentoo.org>

Users of existing pools, especially pools with top-level encrypted datasets, could run into trouble trying to work around Errata #4. Clarify that removing encrypted snapshots and bookmarks is enough to clear the errata. Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Richard Laager <rlaager@wiktel.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Tom Caputi <tcaputi@datto.com> Signed-off-by: Justin Scholz <git@justinscholz.de> Closes openzfs#8682 Closes openzfs#8683

Reviewed-by: Olaf Faaland <faaland1@llnl.gov> Reviewed-by: Richard Laager <rlaager@wiktel.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Justin Scholz <git@justinscholz.de> Closes openzfs#8712 Closes openzfs#8721

After spa_vdev_remove_aux() is called, the config nvlist is no longer valid, as it's been replaced by the new one (with the specified device removed). Therefore any pointers into the nvlist are no longer valid. So we can't save the result of `fnvlist_lookup_string(nv, ZPOOL_CONFIG_PATH)` (in vd_path) across the call to spa_vdev_remove_aux(). Instead, use spa_strdup() to save a copy of the string before calling spa_vdev_remove_aux. Found by AddressSanitizer: ERROR: AddressSanitizer: heap-use-after-free on address ... READ of size 34 at 0x608000a1fcd0 thread T686 #0 0x7fe88b0c166d (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x5166d) #1 0x7fe88a5acd6e in spa_strdup spa_misc.c:1447 #2 0x7fe88a688034 in spa_vdev_remove vdev_removal.c:2259 #3 0x55ffbc7748f8 in ztest_vdev_aux_add_remove ztest.c:3229 #4 0x55ffbc769fba in ztest_execute ztest.c:6714 #5 0x55ffbc779a90 in ztest_thread ztest.c:6761 openzfs#6 0x7fe889cbc6da in start_thread openzfs#7 0x7fe8899e588e in __clone 0x608000a1fcd0 is located 48 bytes inside of 88-byte region freed by thread T686 here: #0 0x7fe88b14e7b8 in __interceptor_free #1 0x7fe88ae541c5 in nvlist_free nvpair.c:874 #2 0x7fe88ae543ba in nvpair_free nvpair.c:844 #3 0x7fe88ae57400 in nvlist_remove_nvpair nvpair.c:978 #4 0x7fe88a683c81 in spa_vdev_remove_aux vdev_removal.c:185 #5 0x7fe88a68857c in spa_vdev_remove vdev_removal.c:2221 openzfs#6 0x55ffbc7748f8 in ztest_vdev_aux_add_remove ztest.c:3229 openzfs#7 0x55ffbc769fba in ztest_execute ztest.c:6714 openzfs#8 0x55ffbc779a90 in ztest_thread ztest.c:6761 openzfs#9 0x7fe889cbc6da in start_thread Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@ixsystems.com> Signed-off-by: Matthew Ahrens <mahrens@delphix.com> Closes openzfs#9706

Under certain loads, the following panic is hit: panic: page fault KDB: stack backtrace: #0 0xffffffff805db025 at kdb_backtrace+0x65 #1 0xffffffff8058e86f at vpanic+0x17f #2 0xffffffff8058e6e3 at panic+0x43 #3 0xffffffff808adc15 at trap_fatal+0x385 #4 0xffffffff808adc6f at trap_pfault+0x4f #5 0xffffffff80886da8 at calltrap+0x8 openzfs#6 0xffffffff80669186 at vgonel+0x186 openzfs#7 0xffffffff80669841 at vgone+0x31 openzfs#8 0xffffffff8065806d at vfs_hash_insert+0x26d openzfs#9 0xffffffff81a39069 at sfs_vgetx+0x149 openzfs#10 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4 openzfs#11 0xffffffff8065a28c at lookup+0x45c openzfs#12 0xffffffff806594b9 at namei+0x259 openzfs#13 0xffffffff80676a33 at kern_statat+0xf3 openzfs#14 0xffffffff8067712f at sys_fstatat+0x2f openzfs#15 0xffffffff808ae50c at amd64_syscall+0x10c openzfs#16 0xffffffff808876bb at fast_syscall_common+0xf8 The page fault occurs because vgonel() will call VOP_CLOSE() for active vnodes. For this reason, define vop_close for zfsctl_ops_snapshot. While here, define vop_open for consistency. After adding the necessary vop, the bug progresses to the following panic: panic: VERIFY3(vrecycle(vp) == 1) failed (0 == 1) cpuid = 17 KDB: stack backtrace: #0 0xffffffff805e29c5 at kdb_backtrace+0x65 #1 0xffffffff8059620f at vpanic+0x17f #2 0xffffffff81a27f4a at spl_panic+0x3a #3 0xffffffff81a3a4d0 at zfsctl_snapshot_inactive+0x40 #4 0xffffffff8066fdee at vinactivef+0xde #5 0xffffffff80670b8a at vgonel+0x1ea openzfs#6 0xffffffff806711e1 at vgone+0x31 openzfs#7 0xffffffff8065fa0d at vfs_hash_insert+0x26d openzfs#8 0xffffffff81a39069 at sfs_vgetx+0x149 openzfs#9 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4 openzfs#10 0xffffffff80661c2c at lookup+0x45c openzfs#11 0xffffffff80660e59 at namei+0x259 openzfs#12 0xffffffff8067e3d3 at kern_statat+0xf3 openzfs#13 0xffffffff8067eacf at sys_fstatat+0x2f openzfs#14 0xffffffff808b5ecc at amd64_syscall+0x10c openzfs#15 0xffffffff8088f07b at fast_syscall_common+0xf8 This is caused by a race condition that can occur when allocating a new vnode and adding that vnode to the vfs hash. If the newly created vnode loses the race when being inserted into the vfs hash, it will not be recycled as its usecount is greater than zero, hitting the above assertion. Fix this by dropping the assertion. FreeBSD-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252700 Reviewed-by: Andriy Gapon <avg@FreeBSD.org> Reviewed-by: Mateusz Guzik <mjguzik@gmail.com> Reviewed-by: Alek Pinchuk <apinchuk@axcient.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Rob Wing <rob.wing@klarasystems.com> Co-authored-by: Rob Wing <rob.wing@klarasystems.com> Submitted-by: Klara, Inc. Sponsored-by: rsync.net Closes openzfs#14501

Andriy Gapon and others added 15 commits March 2, 2016 16:44

zfs_iter_filesystems should iterate both filesystems and volumes

071cb4f

... despite its name.

zfs_fill_zplprops is called with index properties as strings

c3da3de

We have to account for that.

fix userspace validation of utf8only and normalize options

99f556e

zfs_valid_proplist: convert version string to index before checking

88aefa0

zfs_for_each: do not lose earlier errors after later successes

3041f8f

zfs_promote: conflicting snap does not have to be set

fe560ea

E.g. if we try to promote a filesystem that is not a clone.

zfs_stable_ioc_promote: fix unmounting of snapshots

534987d

dmu_objset_find() expects a name that is a filesystem or volume name. It gets confused if given a snapshot name.

lzc_* calls return *the* error code, errno is not it

037324c

if zfs list is called with both -r and -d, then -d should win

7af5a70

allow starting dataset to be filesystem or volume if depth is set

8bdc2ce

Also, do not artificially set ZFS_ITER_DEPTH_LIMIT flag if -d option is not passed. zfs_for_each() already does the right thing without that flag set.

set properties one by one only if we failed to set them at once

1095a0b

Previously the former was done even if the latter succeeded.

atomicity requirement matters only for setting multiple properties

80b6daa

ryao merged commit 43b4583 into ryao:gentoo-zfs-0.6.5.4-release Mar 25, 2016

avg-I deleted the gentoo-zfs-0.6.5.4-release branch September 4, 2019 07:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixes to make zfs test suite happier #4

fixes to make zfs test suite happier #4

avg-I commented Mar 22, 2016

ryao commented Mar 25, 2016

fixes to make zfs test suite happier #4

fixes to make zfs test suite happier #4

Conversation

avg-I commented Mar 22, 2016

ryao commented Mar 25, 2016