Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Target to Extract Binary Artifacts from RPMs #279

Closed

Conversation

adamperlin
Copy link
Contributor

@adamperlin adamperlin commented Jun 4, 2024

What this PR does / why we need it:
There is a desire from some Dalec users to extract the binary artifacts produced as part of Dalec's rpm builds as standalone files. This PR adds a target which produces a single zip file containing the binary artifacts specified in a Dalec spec. Only the binaries are included with no dependencies whatsoever, so it is the user's responsibility to ensure that any standalone binaries extracted with this target are fully statically linked.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #

Special notes for your reviewer:

@adamperlin adamperlin force-pushed the adamperlin/extract-binaries-target branch from 40a8141 to 4546004 Compare June 4, 2024 00:20
@adamperlin adamperlin marked this pull request as ready for review June 4, 2024 00:47
@adamperlin adamperlin requested a review from a team as a code owner June 4, 2024 00:47
@adamperlin
Copy link
Contributor Author

Haven't determined the source of the CI failures yet, it appears to be some kind of timeout

@cpuguy83 cpuguy83 force-pushed the adamperlin/extract-binaries-target branch from a4d095e to d065534 Compare June 4, 2024 19:14
@cpuguy83
Copy link
Member

cpuguy83 commented Jun 4, 2024

Rebased to include #264

cpuguy83 and others added 8 commits June 4, 2024 21:37
The test is using a mariner2 container to test the output of the
windowscross/zip target.
The platform was incorrectly set to use windows/amd64 for the mariner2
container causing the build to fail.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Known issues: does not support binary artifact subpaths, this needs to be fixed
@cpuguy83 cpuguy83 force-pushed the adamperlin/extract-binaries-target branch from d065534 to c47bfa5 Compare June 4, 2024 21:43
@cpuguy83
Copy link
Member

cpuguy83 commented Jun 4, 2024

Rebased this on top of #280 and tests are passing now.

@adamperlin
Copy link
Contributor Author

Rebased this on top of #280 and tests are passing now.

Thank you!!

frontend/azlinux/handler.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
test/bin_extract_test.go Outdated Show resolved Hide resolved
frontend/azlinux/handle_bin.go Outdated Show resolved Hide resolved
Remove zip step from bin extract and output raw binaries in llb.State
Move bin extract tests under testLinuxDistro
Add more descriptive test error messages
Separate handlers for artifacts/binaries and zip into different files and regroup common windows functionality under windows/common.go
@adamperlin adamperlin force-pushed the adamperlin/extract-binaries-target branch from e7df5ca to 017835e Compare June 7, 2024 21:13
frontend/azlinux/handle_bin.go Outdated Show resolved Hide resolved
frontend/azlinux/handle_bin.go Outdated Show resolved Hide resolved
#!/bin/bash
set -e
declare -a RPMS=()
export RPM_BINDIR=$(rpm --eval '%{_bindir}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think (as you mentioned in your commit message) this will not be able to extract executables that aren't in the standard bindir. For example, /usr/libexec (or a subpath of it) will have executable files, and some of the moby packages install files there.

It might be too much of a blunt instrument, but instead of trying to pinpoint the executable files in the rpm, you could just extract everything. Then you can find ./extracted -type f -executable to find the executable files. The downside is you might pick up a script or something that you don't want, but IMO there's no meaningful way to determine ahead of time which "binaries" are desirable vs not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I specifically want this to map to the artifact type in the artifact config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to confirm, are we ok with having this just extract from %{_bindir} (and any subpaths) or do we want to look in any additional locations?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to confirm, are we ok with having this just extract from %{_bindir}

Yes, unless I've misunderstood.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not ignoring this just need some time to think about the edge cases.

Copy link
Contributor

@pmengelbert pmengelbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work on this

@adamperlin
Copy link
Contributor Author

Thanks for the review and sign off @pmengelbert! After a bit of offline discussion today, I think we've decided to put this PR on hold for now as it may not actually be needed to serve the use case

@cpuguy83
Copy link
Member

We discussed this and are thinking this should be handled outside of dalec.
It is difficult to solve all use cases with this and do not want to encourage people to extract the binaries (which are typically going to be dynamically linked anyway).

@adamperlin
Copy link
Contributor Author

Closing this due to above discussion.

@adamperlin adamperlin closed this Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants