Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added shared module for jq template with SBOM generator #85

Merged

Conversation

LaurentGoderre
Copy link
Member

@LaurentGoderre LaurentGoderre commented Oct 2, 2023

Input

{
    name: "erlang",
    version: "25.3.2.6"
    params: {
        os_name: "alpine",
        os_version: "3.17"
    },
    licenses: [
        "Apache-2.0",
        "MPL-2.0"
    ]
} | sbom | tostring

Output:

{
    "spdxVersion": "SPDX-2.3",
    "SPDXID": "SPDXRef-DOCUMENT",
    "name": "erlang-sbom",
    "packages": [
        {
            "name": "erlang",
            "versionInfo": "25.3.2.6",
            "SPDXID": "SPDXRef-Package--erlang",
            "externalRefs": [
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator": "pkg:generic/erlang@25.3.2.6?os_name=alpine&os_version=3.18"
                }
            ],
            "licenseDeclared": "Apache-2.0 AND MPL-2.0"
        }
    ]
}

@codecov-commenter
Copy link

codecov-commenter commented Oct 2, 2023

Codecov Report

Merging #85 (08c9261) into master (6334a4f) will not change coverage.
Report is 2 commits behind head on master.
The diff coverage is n/a.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@           Coverage Diff           @@
##           master      #85   +/-   ##
=======================================
  Coverage   73.10%   73.10%           
=======================================
  Files           7        7           
  Lines         714      714           
=======================================
  Hits          522      522           
  Misses        162      162           
  Partials       30       30           

Copy link

@whalelines whalelines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to added tests for this?

scripts/jq-template.jq Outdated Show resolved Hide resolved
scripts/jq-template.jq Outdated Show resolved Hide resolved
@LaurentGoderre
Copy link
Member Author

@whalelines I am not seeing other tests for scripts. I will go with what @yosifkit recommends

Copy link

@whalelines whalelines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, missed a couple things on my first pass.

scripts/jq-template.jq Outdated Show resolved Hide resolved
referenceLocator: ("pkg:generic/" + .name + "@" + .version + "?" + (.params | [to_entries[] | .key + "=" + .value] | join("\u0026")))
}
],
licenseDeclared: .licenses | join(" AND "),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is licenses required? If not, a conditional should be added to only add the licenseDeclared property if it is not null and not an empty array.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closest I could get is to output "NOASSERTION" when there is none

Copy link
Member

@yosifkit yosifkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I think I'd rather name it something descriptive for what it is, like template-helper-functions.jq. Maybe we'll add a few more useful global helper functions.

I'm still unsure if this is the best place for the logic. Maybe a scanner improvement to scan the Dockerfile and extract this information directly. I guess it'd have to be basically a shell interpreter 🙈, so maybe not easy to implement.

While the Dockerfile is the source of truth for what was installed and from where, this still adds burden to our external maintainers (and will be unlikely to be caught when it isn't updated on a version bump). Most official-images don't use our custom jq templating (even a few we maintain directly don't).


One other item we need to consider is where our tools and scripts will break on a heredoc. In reviewing something else, I ran into https://github.com/docker-library/official-images/blob/8a5c557a6a3fdbe8a83c09ec533bee7d3284f29b/diff-pr.sh#L222-L223 which might have issues with a Dockerfile heredoc. I have to assume that we may have other places where similar assumptions are made.

referenceType: "purl",
referenceLocator: ("pkg:generic/" + .name + "@" + .version + "?" + (.params | [to_entries[] | .key + "=" + .value] | join("\u0026")))
}
],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add downloadLocation if it is given? Are there any other fields that would be useful or interesting to fill in?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I was thinking that would be a next pass

@LaurentGoderre
Copy link
Member Author

LaurentGoderre commented Oct 3, 2023

@yosifkit heredoc aren't even required. The following works

echo '{{
	{
		name: "docker",
		version: .version,
		params: {
			os_name: "alpine",
			os_version: "3.18"
		},
		licenses: [
			"Apache-2.0"
		]
	} | sbom | tostring
}}' > /usr/local/bin/docker.spdx.json

@LaurentGoderre LaurentGoderre merged commit a9fce37 into docker-library:master Oct 3, 2023
6 checks passed
@LaurentGoderre LaurentGoderre deleted the jq-template-functions branch October 3, 2023 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants