You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I implemented SquashFS-support for conda-pack I only did some rudimentary tests regarding the runtime of packing & unpacking with the different compression options of mksquashfs.
I finally got around to running slightly more detailed performance tests.
I ran the perftests on a project that I had lying around with a conda env of size ~1GB:
conda-pack the environment to SquashFS
Mount the packed environment in writeable mode using squashfs + overlayfs.
For reference, Xar uses zstd -Xcompression-level 16 -b 262144 -noI -noX. libzstd comes pre-installed on Ubuntu and Fedora, it's on brew and conda-forge, too.
Unless the maintainers think otherwise I'll open a PR to use zstd with varying levels of -Xcompression-level based on conda-pack --compress-level, only falling back to xz at the highest level. zstd -Xcompression-level 8 seems like a sane default.
The text was updated successfully, but these errors were encountered:
When I implemented SquashFS-support for conda-pack I only did some rudimentary tests regarding the runtime of packing & unpacking with the different compression options of
mksquashfs
.I finally got around to running slightly more detailed performance tests.
I ran the perftests on a project that I had lying around with a conda env of size ~1GB:
conda-pack
the environment to SquashFSsource mountpoint/activate && conda-unpack && pip install -e . && pytest && source mountpoint/deactivate
.conda-pack
noI
: don't compress inode tablenoX
: Don't compress extended attrsFor reference, Xar uses
zstd -Xcompression-level 16 -b 262144 -noI -noX
.libzstd
comes pre-installed on Ubuntu and Fedora, it's on brew and conda-forge, too.Unless the maintainers think otherwise I'll open a PR to use
zstd
with varying levels of-Xcompression-level
based onconda-pack --compress-level
, only falling back toxz
at the highest level.zstd -Xcompression-level 8
seems like a sane default.The text was updated successfully, but these errors were encountered: