Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use recv_fix_encryption_hierarchy for non-recursive send/receive so that encryptionroot is inherited on received side #15687

Open
digitalsignalperson opened this issue Dec 19, 2023 · 4 comments
Labels
Type: Feature Feature request or new feature

Comments

@digitalsignalperson
Copy link

digitalsignalperson commented Dec 19, 2023

Describe the feature would like to see added to OpenZFS

  • explain in zfs send/recv documentation that the encryptionroot property is only preserved when raw sending with -R (for new datasets)
  • preferably: ensure that sending individual datasets with --raw will automatically use the correct encryptionroot on the other side if it exists (like using recv_fix_encryption_hierarchy but for non-recursive sends)

Example of the encryptionroot not preserved

Step 1: Create and encryption root with a couple datasets and replicate it with -R to show we have the same structure on both sides

zfs create -o canmount=off -o mountpoint=none -o encryption=on -o keylocation=prompt -o keyformat=passphrase xpool/enctest
zfs create xpool/enctest/data1
zfs create xpool/enctest/data2
zfs snapshot -r xpool/enctest@snap0
zfs send -R --raw xpool/enctest@snap0 | zfs recv xpool/enctest2

zfs get encryptionroot -r xpool/enctest
NAME                       PROPERTY        VALUE          SOURCE
xpool/enctest              encryptionroot  xpool/enctest  -
xpool/enctest@snap0        encryptionroot  xpool/enctest  -
xpool/enctest/data1        encryptionroot  xpool/enctest  -
xpool/enctest/data1@snap0  encryptionroot  xpool/enctest  -
xpool/enctest/data2        encryptionroot  xpool/enctest  -
xpool/enctest/data2@snap0  encryptionroot  xpool/enctest  -

zfs get encryptionroot -r xpool/enctest2
NAME                        PROPERTY        VALUE           SOURCE
xpool/enctest2              encryptionroot  xpool/enctest2  -
xpool/enctest2@snap0        encryptionroot  xpool/enctest2  -
xpool/enctest2/data1        encryptionroot  xpool/enctest2  -
xpool/enctest2/data1@snap0  encryptionroot  xpool/enctest2  -
xpool/enctest2/data2        encryptionroot  xpool/enctest2  -

As shown by the zfs get encryptionroot, both sides have the expected structure of a single encryption root.

But sending any individual dataset with --raw will cause the received dataset to be it's own encryption root

zfs create xpool/enctest/data3
zfs snapshot -r xpool/enctest/data3@data3
zfs send --raw xpool/enctest/data3@data3 | zfs recv xpool/enctest2/data3

zfs get encryptionroot -r xpool/enctest2
NAME                        PROPERTY        VALUE                 SOURCE
xpool/enctest2              encryptionroot  xpool/enctest2        -
xpool/enctest2@snap0        encryptionroot  xpool/enctest2        -
xpool/enctest2/data1        encryptionroot  xpool/enctest2        -
xpool/enctest2/data1@snap0  encryptionroot  xpool/enctest2        -
xpool/enctest2/data2        encryptionroot  xpool/enctest2        -
xpool/enctest2/data2@snap0  encryptionroot  xpool/enctest2        -
xpool/enctest2/data3        encryptionroot  xpool/enctest2/data3  -
xpool/enctest2/data3@data3  encryptionroot  xpool/enctest2/data3  -

This can be fixed with loading all the keys and doing zfs change-key -i although I see some bugs related to this to watch out for. Edit: I tested this out and documented my results here #12123 (comment) and I didn't encounter and issues.

But zfs change-key -i is annoying if you just received 100+ datasets. You would need to write a script to supply the password to each dataset (zfs load-key prompts for every single dataset).

Note that as long as the encryptionroot gets set correctly once, then future non -R incremental sends seem to keep the correct encryptionroot.

zfs snapshot xpool/enctest/data1@snap1
zfs send --raw -i @snap0 xpool/enctest/data1@snap1 | zfs recv xpool/enctest2/data1

zfs get encryptionroot -r xpool/enctest2
NAME                        PROPERTY        VALUE                 SOURCE
xpool/enctest2              encryptionroot  xpool/enctest2        -
xpool/enctest2@snap0        encryptionroot  xpool/enctest2        -
xpool/enctest2/data1        encryptionroot  xpool/enctest2        -
xpool/enctest2/data1@snap0  encryptionroot  xpool/enctest2        -
xpool/enctest2/data1@snap1  encryptionroot  xpool/enctest2        -
xpool/enctest2/data2        encryptionroot  xpool/enctest2        -
xpool/enctest2/data2@snap0  encryptionroot  xpool/enctest2        -
xpool/enctest2/data3        encryptionroot  xpool/enctest2/data3  -
xpool/enctest2/data3@data3  encryptionroot  xpool/enctest2/data3  -

How will this feature improve OpenZFS?

  • Make it clear how encryptionroot is set on the received side.
  • Make it easier to send a single dataset within an encryptionroot without needing to -R send the whole thing at any given time.
  • Avoid needing to do zfs change-key -i on every received dataset if it came from the same original encryption root
@digitalsignalperson
Copy link
Author

digitalsignalperson commented Dec 22, 2023

"The root of a raw send should never be force-inherited." from

strcmp(top_zfs, fsname) != 0) {

/*
* If the dataset is not flagged as an encryption root and is
* currently an encryption root, force it to inherit from its
* parent. The root of a raw send should never be
* force-inherited.
*/

except in my above example if I did zfs send -R --raw xpool/enctest/data3@data3 | zfs recv xpool/enctest2/data3 I actually do want the root of the raw send to be force-inherited.

This change was introduced here, but I couldn't find reference to an issue or why.
4807c0b
#6595

-		if (!stream_encroot && is_encroot) {
+		if (!stream_encroot && is_encroot &&
+		    strcmp(top_zfs, fsname) != 0) {

Can there be an option on zfs recv to allow this ( -x encryptionroot)?
Or is the intention detectable?
What is "The root of a raw send should never be force-inherited." protecting against?

And similarly for non-recursive, can zfs_receive_one() call lzc_change_key(fsname, DCP_CMD_FORCE_INHERIT, if applicable or specified?

@digitalsignalperson
Copy link
Author

This code comment could be a bug or lead to one

* will fixup the keylocation anyway, so we temporarily unset

in zfs_receive_one() this comment states

		/*
		 * The keylocation property may only be set on encryption roots,
		 * but this dataset might not become an encryption root until
		 * recv_fix_encryption_hierarchy() is called. That function
		 * will fixup the keylocation anyway, so we temporarily unset
		 * the keylocation for now to avoid any errors from the receive
		 * ioctl.
		 */

but " That function will fixup the keylocation anyway" is not true: recv_fix_encryption_hierarchy() is never called from zfs_receive_one() as implied.

  • zfs_receive_impl() will call either zfs_receive_one() or zfs_receive_package().
  • recv_fix_encryption_hierarchy() is only called from zfs_receive_package()

patch attempt

I tried reverting the chunk from 4807c0b #6595 that prevents force-inheriting the root of a raw send. Revert patch:

--- libzfs_sendrecv.c
+++ libzfs_sendrecv.c
@@ -3472,11 +3472,9 @@
 		/*
 		 * If the dataset is not flagged as an encryption root and is
 		 * currently an encryption root, force it to inherit from its
-		 * parent. The root of a raw send should never be
-		 * force-inherited.
+		 * parent.
 		 */
-		if (!stream_encroot && is_encroot &&
-		    strcmp(top_zfs, fsname) != 0) {
+		if (!stream_encroot && is_encroot {
 			err = lzc_change_key(fsname, DCP_CMD_FORCE_INHERIT,
 			    NULL, NULL, 0);
 			if (err != 0) {

But doing a zfs send -R --raw xpool/enc1/data@snap | zfs recv xpool/enc2/data, the result still has encryption root set to xpool/enc2/data and not xpool/enc2

@digitalsignalperson digitalsignalperson changed the title Explain in zfs send documentation that encryptionroot is only preserved for -R and/or add a feature to always preserve it. Use recv_fix_encryption_hierarchy for non-recursive send/receive so that encryptionroot is inherited on received side Dec 31, 2023
@digitalsignalperson
Copy link
Author

Oops so that patch above had a typo, and it also worked as intended (I was only building zfs-dkms and not zfs-utils), but isn't the whole story. If we receive into a copy of the encryption root, we now inherit as expected. But if we receive into a different independent encryption root, it will still inherit when it should not.

What is needed is, before force inherit, detect if the candidate encryption root has the same key (? or ivset or something) as the current dataset. I don't know my way around enough to know how this can be looked up.

Details below.

Fix missing parenthesis:

--- libzfs_sendrecv.c
+++ libzfs_sendrecv.c
@@ -3472,11 +3472,9 @@
 		/*
 		 * If the dataset is not flagged as an encryption root and is
 		 * currently an encryption root, force it to inherit from its
-		 * parent. The root of a raw send should never be
-		 * force-inherited.
+		 * parent.
 		 */
-		if (!stream_encroot && is_encroot &&
-		    strcmp(top_zfs, fsname) != 0) {
+		if (!stream_encroot && is_encroot) {
 			err = lzc_change_key(fsname, DCP_CMD_FORCE_INHERIT,
 			    NULL, NULL, 0);
 			if (err != 0) {

Test code:

# Create test pool
dd if=/dev/zero of=/root/zpool bs=1M count=128
zpool create testpool /root/zpool

# Create encryptionroot and some datasets
echo "12345678" | zfs create -o canmount=off -o mountpoint=/mnt -o encryption=on -o keylocation=prompt -o keyformat=passphrase testpool/enc
zfs create testpool/enc/data1
zfs create testpool/enc/data2
touch /mnt/data1/x
touch /mnt/data2/x
zfs snapshot -r testpool/enc@1

# Make a recursive copy of the encryption root
zfs send -Rw testpool/enc@1 | zfs recv testpool/enc_copy

# Make a new dataset on the original encryption root and try to send to the new one
zfs create testpool/enc/data3
touch /mnt/data3/x
zfs snapshot testpool/enc/data3@x
zfs send -Rw testpool/enc/data3@x | zfs recv testpool/enc_copy/data3

# Expected behavior: the received encryption root will be enc_copy 

Before the fix:

$zfs get encryptionroot

NAME                       PROPERTY        VALUE                    SOURCE
testpool                   encryptionroot  -                        -
testpool/enc               encryptionroot  testpool/enc             -
testpool/enc@1             encryptionroot  testpool/enc             -
testpool/enc/data1         encryptionroot  testpool/enc             -
testpool/enc/data1@1       encryptionroot  testpool/enc             -
testpool/enc/data2         encryptionroot  testpool/enc             -
testpool/enc/data2@1       encryptionroot  testpool/enc             -
testpool/enc/data3         encryptionroot  testpool/enc             -
testpool/enc/data3@x       encryptionroot  testpool/enc             -
testpool/enc_copy          encryptionroot  testpool/enc_copy        -
testpool/enc_copy@1        encryptionroot  testpool/enc_copy        -
testpool/enc_copy/data1    encryptionroot  testpool/enc_copy        -
testpool/enc_copy/data1@1  encryptionroot  testpool/enc_copy        -
testpool/enc_copy/data2    encryptionroot  testpool/enc_copy        -
testpool/enc_copy/data2@1  encryptionroot  testpool/enc_copy        -
testpool/enc_copy/data3    encryptionroot  testpool/enc_copy/data3  -
testpool/enc_copy/data3@x  encryptionroot  testpool/enc_copy/data3  -

After the fix:

$zfs get encryptionroot

NAME                       PROPERTY        VALUE              SOURCE
testpool                   encryptionroot  -                  -
testpool/enc               encryptionroot  testpool/enc       -
testpool/enc@1             encryptionroot  testpool/enc       -
testpool/enc/data1         encryptionroot  testpool/enc       -
testpool/enc/data1@1       encryptionroot  testpool/enc       -
testpool/enc/data2         encryptionroot  testpool/enc       -
testpool/enc/data2@1       encryptionroot  testpool/enc       -
testpool/enc/data3         encryptionroot  testpool/enc       -
testpool/enc/data3@x       encryptionroot  testpool/enc       -
testpool/enc_copy          encryptionroot  testpool/enc_copy  -
testpool/enc_copy@1        encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data1    encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data1@1  encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data2    encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data2@1  encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data3    encryptionroot  testpool/enc_copy  -
testpool/enc_copy/data3@x  encryptionroot  testpool/enc_copy  -

and we can mount and access the data

$ zfs load-key -r testpool/enc_copy  # only prompted ONCE for password
$ zfs mount -a
$ find /mnt/zpool | sort | grep data3
/mnt/zpool/enc/data3
/mnt/zpool/enc/data3/x
/mnt/zpool/enc_copy/data3
/mnt/zpool/enc_copy/data3/x

This is good. But what is now broken is if we receive in to a different independent encryption root, it will force inherit anyway.

# Create a new independent encryption root
$ echo "12345678" | zfs create -o canmount=off -o mountpoint=/mnt -o encryption=on -o keylocation=prompt -o keyformat=passphrase testpool/enc_other

# Send one dataset from the other encryption root
# Expected result: since they are independent encryption roots, do not force inherit
# Actual result: it force inherits anyway
$ zfs send -Rw testpool/enc/data3@x | zfs recv testpool/enc_other/data3
cannot mount 'testpool/enc_other/data3': Permission denied

$ zfs get encryptionroot -r testpool/enc_other
NAME                        PROPERTY        VALUE               SOURCE
testpool/enc_other          encryptionroot  testpool/enc_other  -
testpool/enc_other/data3    encryptionroot  testpool/enc_other  -
testpool/enc_other/data3@x  encryptionroot  testpool/enc_other  -

@digitalsignalperson
Copy link
Author

I just discovered that pyzfs exposes the force_inherit function! So now we can at least fix the encryptionroot manually if we know what we're doing.

# Create test pool
dd if=/dev/zero of=/root/zpool bs=1M count=128
zpool create testpool /root/zpool

# Create encryptionroot and some datasets
echo "12345678" | zfs create -o canmount=off -o mountpoint=/mnt -o encryption=on -o keylocation=prompt -o keyformat=passphrase testpool/enc
zfs create testpool/enc/data1
zfs create testpool/enc/data2
touch /mnt/data1/x
touch /mnt/data2/x
zfs snapshot -r testpool/enc@1

# Make a recursive copy of the encryption root
zfs send -Rw testpool/enc@1 | zfs recv testpool/enc_copy

# Make a new dataset on the original encryption root and try to send to the new one
zfs create testpool/enc/data3
touch /mnt/data3/x
zfs snapshot testpool/enc/data3@x
zfs send -Rw testpool/enc/data3@x | zfs recv testpool/enc_copy/data3

# Fix the encryptionroot ourselves:
python -c 'import libzfs_core; libzfs_core.lzc_change_key(b"testpool/enc_copy/data3", "force_inherit")'

digitalsignalperson added a commit to digitalsignalperson/zfs that referenced this issue Jan 25, 2024
This implements functionality already available in pyzfs
i.e. libzfs_core.lzc_change_key(b"tank/enc/data", "force_inherit")
Unlike zfs change-key -i, the key is not required to be loaded.
Work-around to openzfs#15687.

Signed-off-by: andy <andy@digitalsignalperson.com>
digitalsignalperson added a commit to digitalsignalperson/zfs that referenced this issue Jan 25, 2024
This implements functionality already available in pyzfs
i.e. libzfs_core.lzc_change_key(b"tank/enc/data", "force_inherit")
Unlike zfs change-key -i, the key is not required to be loaded.
Work-around to openzfs#15687.

Signed-off-by: andy <andy@digitalsignalperson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Feature Feature request or new feature
Projects
None yet
Development

No branches or pull requests

1 participant