RFC: improve initramfs image performance and efficiency via cpio reflinks

[ddiss]

This patchset attempts to speed up initramfs generation for some common (Btrfs / XFS) setups by having Dracut make heavier use of reflinks (AKA copy-on-write clones) during initramfs generation. A good portion of an uncompressed+unstripped initramfs image is duplicate data, which really shouldn't need to be shuffled around when on the same COW clone capable FS.

Dracut already uses cp --reflink=auto when shuffling most things into the temporary staging area. This patchset adds a new --cpio-reflink flag which sees Dracut call GNU cpio using the new --reflink and --chain parameters. Support for these new parameters can be found in my cpio repository at https://github.com/ddiss/cpio/tree/copy_file_range_2_13 .
As XFS and Btrfs require filesystem alignment for extent sharing, a new padcpio utility is added to provide padding of file data to a block-size boundary.

This allows for:

• improved space efficiency
  • initramfs creation avoids data duplication
• improved performance
  • initramfs image needn't be stripped / compressed / decompressed

The following caveats would be present for dracut to successfully use reflink (otherwise fallback to read/write):

• root, boot and dracut staging (/var/tmp) exist on the same Btrfs or XFS filesystem
• paths don't have nocow flags set
• boot performance may be negatively affected by fragmentation, but that should be compensated by the removal of compression / decompression

Benchmarks are still ongoing, but my initial results show:

---------------------------------+----------+----------+-----------
     Benchmark                   |  Before  |  After   |  Change
---------------------------------+----------+----------+-----------
Dracut create image runtime      |  8.452s  |  7.635s  |  -9.666%
---------------------------------+----------+----------+-----------
initramfs data (fiemap)          |          |          |
- total                          | 12894208 | 34009088 |  +163.7%
- shared (dedup)                 |     0    | 24068096 |
- exclusive                      | 12894208 |  9940992 |  -22.90%
---------------------------------+----------+----------+-----------
QEMU cold boot to Dracut init    |  3.208s  |  2.850s  |  -11.15%
---------------------------------+----------+----------+-----------

See #1141 (comment) for details.

RFC: The top commit of this patchset shouldn't be merged until GNU cpio --reflink support is upstream. I believe all other changes are read.

Fixes: #1141

[johannbg]

You seem to be missing shell competition and documentation ( man ) update/page from the PR.

[haraldh]

So, the whole thing is most interesting for initramfs images stored on xfs/btrfs for qemu booting, as moving it to EFI vfat will remove the dedup feature. Boot times for bare metal would also increase due to non-compression.

Did I understand this correctly?

[ddiss]

So, the whole thing is most interesting for initramfs images stored on xfs/btrfs for qemu booting, as moving it to EFI vfat will remove the dedup feature. Boot times for bare metal would also increase due to non-compression.

Did I understand this correctly?

Yes, correct, it's most beneficial for environments where initramfs source files, Dracut staging area and destination image are all on the same XFS or Btrfs filesystem. If the Dracut destination is EFI vfat then duplication is expected, although it would still allow for $initdir/*->${DRACUT_TMPDIR}/initramfs.img cpio data copy I/O to be reduced.

[haraldh]

I am sorry, you have to rebase due to upstream changes in Makefile.

[ddiss]

I am sorry, you have to rebase due to upstream changes in Makefile.

No problem 😃 . I've made some extra changes to use statfs reported optimal alignment for padding, rather than the hardcoded 4K. I'll force push here along with @johannbg 's suggested changes when done testing.

[johannbg]

@ddiss can you give us a link to your cpio --reflink upstream PR, looking upstream and based on lack of activity there,it looks like it can literally take years before your PR gets merged and is included in a new release.

[ddiss]

@ddiss can you give us a link to your cpio --reflink upstream PR, looking upstream and based on lack of activity there,it looks like it can literally take years before your PR gets merged and is included in a new release.

They've now been submitted via https://savannah.gnu.org/patch/index.php?10044 and the upstream mailing list:
https://lists.gnu.org/archive/html/bug-cpio/2021-03/msg00008.html (edit: fix ML link)

[ddiss]

Changes since previous version:

• rebase against current master branch
• use statfs to determine optimal padding alignment
• address @johannbg 's review feedback
• add minor test cleanup: d139603
• improve man-page documentation

[ddiss]

Changes since previous version:

• rebase against current master branch
• apply astyle indent rules to padcpio

[ddiss]

Changes since previous version:

• add padcpio to dracut.spec

[ddiss]

Changes since previous version:

• change TEST-62-CPIO commit message to make Commisery happy

[ddiss]

So, the whole thing is most interesting for initramfs images stored on xfs/btrfs for qemu booting, as moving it to EFI vfat will remove the dedup feature. Boot times for bare metal would also increase due to non-compression.
Did I understand this correctly?

Yes, correct, it's most beneficial for environments where initramfs source files, Dracut staging area and destination image are all on the same XFS or Btrfs filesystem. If the Dracut destination is EFI vfat then duplication is expected

One thing I forgot to clear up here is that these changes don't just target qemu workloads - [open]SUSE distros commonly carry initramfs with the root filesystem, so that system rollback (from a Btrfs snapshot) is possible.

[haraldh]

I will merge this, if cpio upstream supports it.

[johannbg]

@ddiss given that it can take a while for upstream cpio to merge this would you mind putting this into draft mode if and then when it finally gets merged take it out of draft mode?

[ddiss]

I will merge this, if cpio upstream supports it.

Thanks Harald. I'll post an update to the cpio patchset this week following @luis-henrix 's review (https://lists.gnu.org/archive/html/bug-cpio/2021-03/msg00008.html).

@ddiss given that it can take a while for upstream cpio to merge this would you mind putting this into draft mode if and then when it finally gets merged take it out of draft mode?

Sure, that works for me. I'm probably missing something, but I don't see a draft-mode button in the Github UI. Are you able to do it from your side?

[johannbg]

@ddiss done, it's a link to click not a visible button ( for whatever reason ) and the link is in the bottom right corner, in the reviewers sections

[ddiss]

Just a minor update: I reworked and posted a new version of the GNU cpio patchset to the list last week following Luis' review - https://lists.gnu.org/archive/html/bug-cpio/2021-04/threads.html .

[johannbg]

Looks like this wont be reviewed until sometime in May based on Sergey's response so this probably will end up in the 55 release ( probably sometime in June ) or the 56 release ( August/September ) or the release after cpio gets a new release.

[ddiss]

After seeing Rust's native support for copy_file_range() via fs::copy(), I went ahead and wrote a prototype dracut-cpio utility:
https://github.com/ddiss/dracut/tree/dracut_cpio

[haraldh]

After seeing Rust's native support for copy_file_range() via fs::copy(), I went ahead and wrote a prototype dracut-cpio utility:
https://github.com/ddiss/dracut/tree/dracut_cpio

Here is my (outdated) cp version in rust for dracut-install with sparse support:
https://github.com/haraldh/dracut-install/blob/master/src/file/mod.rs#L228

[haraldh]

On a funny side note, part of fs::copy() was one of my first rust project contributions :)

Additional sparse file support was rejected.

[ddiss]

Here is my (outdated) cp version in rust for dracut-install with sparse support:
https://github.com/haraldh/dracut-install/blob/master/src/file/mod.rs#L228

Nice! I wasn't aware of your work in this area. I'd also be happy to integrate the dracut-cpio functionality into dracut-install.rs directly if you plan on continuing in that direction.

[ddiss]

After seeing Rust's native support for copy_file_range() via fs::copy(), I went ahead and wrote a prototype dracut-cpio utility:
https://github.com/ddiss/dracut/tree/dracut_cpio

I've updated the dracut-cpio patch to the point where it can now be used for proper testing as a swap-in replacement for GNU cpio. Like my GNU cpio patchset, it supports cpio reflinks and with data segment alignment. Data segment alignment is handled in a much cleaner fashion by injecting extra zeros into the cpio archive filename fields (rather than adding extra pad files to initramfs).
GNU cpio and kernel seem to handle filename zero-injection just fine, but I need to do some further testing and benchmarking.

[stale]

This issue is being marked as stale because it has not had any recent activity. It will be closed if no further activity occurs. If this is still an issue in the latest release of Dracut and you would like to keep it open please comment on this issue within the next 7 days. Thank you for your contributions.