Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support archives as package src, autonamespace and other improvements #4

Merged
merged 11 commits into from
Nov 29, 2021

Conversation

puerco
Copy link
Member

@puerco puerco commented Nov 29, 2021

What type of PR is this?

/kind bug
/kind cleanup
/kind feature
/kind deprecation

What this PR does / why we need it:

This PR adds a bunch of improvements to the bom tool and the spdx libraries:

  • Adds a few more unit tests to the spdx package to cover the following functions:
    • spdx.GetImageReferences
    • spdx.TestPullImagesToArchive
    • spdx.TestGetDirectoryTree
    • spdx.TestIgnorePatterns
  • The --tarballs flag is now deprecated. It has been replaced with --image-archive during demos and chats, it proved to be confusing (it still works but will print a warning)
  • There is a new flag --archive that adds archives (currently tars) as packages. Its files are license-scanned and listed in the package
  • Passing a flag defining the SPDX document namespace is not required anymore. The generator now defines it using the spdx.org public URL defined in the spec

Which issue(s) this PR fixes:

Special notes for your reviewer:

Replay of kubernetes/release#2319 on k-sigs/bom
/cc @kubernetes/release-engineering @kubernetes/release-managers

Does this PR introduce a user-facing change?

* Added a few more unit tests to the `spdx` package to cover the following functions: `spdx.GetImageReferences` `spdx.TestPullImagesToArchive` `spdx.TestGetDirectoryTree` `spdx.TestIgnorePatterns`
* bom: The `--tarballs` flag is now deprecated. It has been replaced with `--image-archive` during demos and chats, it proved to be confusing (it still works but will print a warning)
* bom: There is a new flag: `--archive`. When enabled, bom adds archives (currently tars) as spdx packages to the doc. Its files are license-scanned and listed in the package
* bom: Passing a flag defining the SPDX document namespace is not required anymore. The generator now defines it using the spdx.org public URL defined in the 2.2+ spec.
* The spdx package now supports reading compressed tars 

This commit modifies the spdx.PackageFromLayerTarball to make it generic.
The idea is to be able to generate packages from any tarbal, not just
layers.
This commit adds more testing to the spdx package which currently has
a low coverage. These new tests sover the following functions:

 spdx.GetImageReferences
 spdx.TestPullImagesToArchive
 spdx.TestGetDirectoryTree
 spdx.TestIgnorePatterns

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
This commit modifies the method signatures of the SPDX package building
funcs in the implementation to make their signatures alike: recive an
options struct as the first arg and then each package type data.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
The spdx package now supports compressed tar files when creating
packages from tarfiles or container images.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
This commit moves the spdx.PackageFromDirectory() to the implementation
to make it mockable.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
SPDX objects now support an option to add prefixes to identifiers. The
purpose of the prefix is to add an arbitrary string to IDs to help avoid
colisions when generating identifiers.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
This commit deprecates the `bom generate --tarball` and renames it to
`--image-archive`. The old flag still works but it is marked as deprecated.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
bom and the underlying lubraries now support generating packages from
archives. Currently only tar|tar.gz files are supported but more can be
added in the future.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
Previously, we required the SPDX document namespace to be supplied
in a config file or a flag. This commits adds support to automatically
generate the namespace using the https://spdx.org/spdxdocs/ URL as
described in the SPDX spec.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
When autogenerating a SPDX ID, we now preserve the special
chars in seed strings to avoid reducing them to a point where
collisions start to happen.

Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
@k8s-ci-robot
Copy link
Contributor

@puerco: GitHub didn't allow me to request PR reviews from the following users: kubernetes/release-engineering, kubernetes/release-managers.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

What type of PR is this?

/kind bug
/kind cleanup
/kind feature
/kind deprecation

What this PR does / why we need it:

This PR adds a bunch of improvements to the bom tool and the spdx libraries:

  • Adds a few more unit tests to the spdx package to cover the following functions:
  • spdx.GetImageReferences
  • spdx.TestPullImagesToArchive
  • spdx.TestGetDirectoryTree
  • spdx.TestIgnorePatterns
  • The --tarballs flag is now deprecated. It has been replaced with --image-archive during demos and chats, it proved to be confusing (it still works but will print a warning)
  • There is a new flag --archive that adds archives (currently tars) as packages. Its files are license-scanned and listed in the package
  • Passing a flag defining the SPDX document namespace is not required anymore. The generator now defines it using the spdx.org public URL defined in the spec

Which issue(s) this PR fixes:

Special notes for your reviewer:

Replay of kubernetes/release#2319 on k-sigs/bom
/cc @kubernetes/release-engineering @kubernetes/release-managers

Does this PR introduce a user-facing change?

* Added a few more unit tests to the `spdx` package to cover the following functions: `spdx.GetImageReferences` `spdx.TestPullImagesToArchive` `spdx.TestGetDirectoryTree` `spdx.TestIgnorePatterns`
* bom: The `--tarballs` flag is now deprecated. It has been replaced with `--image-archive` during demos and chats, it proved to be confusing (it still works but will print a warning)
* bom: There is a new flag: `--archive`. When enabled, bom adds archives (currently tars) as spdx packages to the doc. Its files are license-scanned and listed in the package
* bom: Passing a flag defining the SPDX document namespace is not required anymore. The generator now defines it using the spdx.org public URL defined in the 2.2+ spec.
* The spdx package now supports reading compressed tars 

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/feature Categorizes issue or PR as related to a new feature. kind/deprecation Categorizes issue or PR as related to a feature/enhancement marked for deprecation. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 29, 2021
Copy link
Member

@cpanato cpanato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
thanks for that!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 29, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, puerco

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 6434978 into kubernetes-sigs:main Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/deprecation Categorizes issue or PR as related to a feature/enhancement marked for deprecation. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants