Use pzstd for FILESYSTEM_DOCKERFS when defined BUILD_REDUCE_IMAGE_SIZE #21824
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Fix zstd dockerfs build for ONIE and for non-amd64
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Currently, there are two methods of having a reduced image size: BUILD_REDUCE_IMAGE_SIZE build configuration (resulting in the use of a zstd-compressed docker tarball for the installation file), and docker_inram (resulting in the docker tarball being extracted into RAM at bootup time instead to disk) (this particular option seems to be better supported on Aboot and not ONIE). These two are logically separate; one is a build-time configuration for an image, and the other is a runtime platform-specific option. This means that it's theoretically possible to have an image be built with BUILD_REDUCE_IMAGE_SIZE enabled and then be loaded onto a device that doesn't use docker_inram. If the image is being installed from SONiC, then that's _probably_ file, as long as the SONiC installation has zstd installed. If the image is being installed from ONIE or Aboot, then both of those environments currently don't support zstd-compressed tarballs, so image installation fails. Because of that, fail explicitly saying that this image is not supported in that environment. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Add checks for zstd tarball extraction from ONIE or Aboot
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Fix references to gzip/zstd variables
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines successfully started running 1 pipeline(s). |
11 tasks
mssonicbld
added a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Feb 26, 2025
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
Same with sonic-net#21824
Enabled the docker inram feature for slim image.
It would extract the docker image to ram during the boot, so it would take extra times during boot.
##### Work item tracking
- Microsoft ADO **(number only)**:
31323281
#### How I did it
Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time.
Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed.
We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that.
So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file.
Plan to support zstd for all types of images in the future to unify the docker file image .
#### How to verify it
Boot swi slim image and normal image, boot mellanox bin image, all boot successfully.
<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
- [ ] 202211
- [ ] 202305
#### Tested branch (Please provide the tested image version)
<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->
- [ ] <!-- image version 1 -->
- [ ] <!-- image version 2 -->
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
<!--
Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->
#### A picture of a cute animal (not mandatory but encouraged)
11 tasks
bingwang-ms
pushed a commit
that referenced
this pull request
Feb 27, 2025
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
Same with #21824
Enabled the docker inram feature for slim image.
It would extract the docker image to ram during the boot, so it would take extra times during boot.
##### Work item tracking
- Microsoft ADO **(number only)**:
31323281
#### How I did it
Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time.
Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed.
We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that.
So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file.
Plan to support zstd for all types of images in the future to unify the docker file image .
#### How to verify it
Boot swi slim image and normal image, boot mellanox bin image, all boot successfully.
<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
- [ ] 202211
- [ ] 202305
#### Tested branch (Please provide the tested image version)
<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->
- [ ] <!-- image version 1 -->
- [ ] <!-- image version 2 -->
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
<!--
Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->
#### A picture of a cute animal (not mandatory but encouraged)
mssonicbld
added a commit
to mssonicbld/sonic-buildimage
that referenced
this pull request
Mar 11, 2025
<!--
Please make sure you've read and understood our contributing guidelines:
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
** Make sure all your commits include a signature generated with `git commit -s` **
If this is a bug fix, make sure your description includes "fixes #xxxx", or
"closes #xxxx" or "resolves #xxxx"
Please provide the following information:
-->
#### Why I did it
Same with sonic-net#21824
Enabled the docker inram feature for slim image.
It would extract the docker image to ram during the boot, so it would take extra times during boot.
##### Work item tracking
- Microsoft ADO **(number only)**:
31323281
#### How I did it
Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time.
Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed.
We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that.
So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file.
Plan to support zstd for all types of images in the future to unify the docker file image .
#### How to verify it
Boot swi slim image and normal image, boot mellanox bin image, all boot successfully.
<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->
#### Which release branch to backport (provide reason below if selected)
<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->
- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
- [ ] 202211
- [ ] 202305
#### Tested branch (Please provide the tested image version)
<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->
- [ ] <!-- image version 1 -->
- [ ] <!-- image version 2 -->
#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->
<!--
Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->
#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->
#### A picture of a cute animal (not mandatory but encouraged)
11 tasks
mssonicbld
added a commit
that referenced
this pull request
Mar 12, 2025
<!-- Please make sure you've read and understood our contributing guidelines: https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md failure_prs.log skip_prs.log Make sure all your commits include a signature generated with `git commit -s` ** If this is a bug fix, make sure your description includes "fixes #xxxx", or "closes #xxxx" or "resolves #xxxx" Please provide the following information: --> #### Why I did it Same with #21824 Enabled the docker inram feature for slim image. It would extract the docker image to ram during the boot, so it would take extra times during boot. ##### Work item tracking - Microsoft ADO **(number only)**: 31323281 #### How I did it Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time. Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed. We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that. So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file. Plan to support zstd for all types of images in the future to unify the docker file image . #### How to verify it Boot swi slim image and normal image, boot mellanox bin image, all boot successfully. <!-- If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012. --> #### Which release branch to backport (provide reason below if selected) <!-- - Note we only backport fixes to a release branch, *not* features! - Please also provide a reason for the backporting below. - e.g. - [x] 202006 --> - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 - [ ] 202211 - [ ] 202305 #### Tested branch (Please provide the tested image version) <!-- - Please provide tested image version - e.g. - [x] 20201231.100 --> - [ ] <!-- image version 1 --> - [ ] <!-- image version 2 --> #### Description for the changelog <!-- Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> <!-- Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU. --> #### Link to config_db schema for YANG module changes <!-- Provide a link to config_db schema for the table for which YANG model is defined Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md --> #### A picture of a cute animal (not mandatory but encouraged)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I did it
We enabled the docker inram feature for slim image.
It would extract the docker image to ram during the boot, so it would take extra times during boot.
Work item tracking
31323281
How I did it
Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time.
How to verify it
Boot image on 7060 and also run warm-reboot pipeline on 7060.
Which release branch to backport (provide reason below if selected)
Tested branch (Please provide the tested image version)
Description for the changelog
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)