Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable libexec artifacts #408

Merged
merged 9 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions artifacts.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import (
type Artifacts struct {
// Binaries is the list of binaries to include in the package.
Binaries map[string]ArtifactConfig `yaml:"binaries,omitempty" json:"binaries,omitempty"`
// Libexec is the list of additional binaries that may be invoked by the main package binary.
Libexec map[string]ArtifactConfig `yaml:"libexec,omitempty" json:"libexec,omitempty"`
// Manpages is the list of manpages to include in the package.
Manpages map[string]ArtifactConfig `yaml:"manpages,omitempty" json:"manpages,omitempty"`
// DataDirs is a list of read-only architecture-independent data files, to be placed in /usr/share/
Expand Down
7 changes: 7 additions & 0 deletions docs/spec.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,13 @@
"type": "object",
"description": "Binaries is the list of binaries to include in the package."
},
"libexec": {
"additionalProperties": {
"$ref": "#/$defs/ArtifactConfig"
},
"type": "object",
"description": "Libexec is the list of additional binaries that may be invoked by the main package binary."
},
"manpages": {
"additionalProperties": {
"$ref": "#/$defs/ArtifactConfig"
Expand Down
11 changes: 11 additions & 0 deletions frontend/deb/debroot.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
_ "embed"

"github.com/Azure/dalec"
"github.com/Azure/dalec/frontend"
"github.com/Azure/dalec/frontend/pkg/bkfs"
"github.com/moby/buildkit/client/llb"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
Expand Down Expand Up @@ -452,6 +453,16 @@ func createInstallScripts(worker llb.State, spec *dalec.Spec, dir string) []llb.
}
}

if len(spec.Artifacts.Libexec) > 0 {
sorted := dalec.SortMapKeys(spec.Artifacts.Libexec)
for _, key := range sorted {
cfg := spec.Artifacts.Libexec[key]
resolved := cfg.ResolveName(key)
subPath := frontend.DefaultLibexecSubpath(spec, key)
writeInstall(key, filepath.Join("/usr/libexec", subPath), resolved)
}
}

if len(spec.Artifacts.Libs) > 0 {
sorted := dalec.SortMapKeys(spec.Artifacts.Libs)
for _, key := range sorted {
Expand Down
23 changes: 23 additions & 0 deletions frontend/rpm/template.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"text/template"

"github.com/Azure/dalec"
"github.com/Azure/dalec/frontend"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
)
Expand Down Expand Up @@ -499,6 +500,18 @@ func (w *specWrapper) Install() fmt.Stringer {
}
}

if w.Spec.Artifacts.Libexec != nil {
cpuguy83 marked this conversation as resolved.
Show resolved Hide resolved
libexecFileKeys := dalec.SortMapKeys(w.Spec.Artifacts.Libexec)
for _, k := range libexecFileKeys {
le := w.Spec.Artifacts.Libexec[k]
leCopy := dalec.ArtifactConfig{
SubPath: frontend.DefaultLibexecSubpath(w.Spec, k),
Name: le.Name,
}
copyArtifact(`%{buildroot}/%{_libexecdir}`, k, &leCopy)
}
}

configKeys := dalec.SortMapKeys(w.Spec.Artifacts.ConfigFiles)
for _, c := range configKeys {
cfg := w.Spec.Artifacts.ConfigFiles[c]
Expand Down Expand Up @@ -599,6 +612,16 @@ func (w *specWrapper) Files() fmt.Stringer {
}
}

if w.Spec.Artifacts.Libexec != nil {
dataKeys := dalec.SortMapKeys(w.Spec.Artifacts.Libexec)
for _, k := range dataKeys {
df := w.Spec.Artifacts.Libexec[k]
subPath := frontend.DefaultLibexecSubpath(w.Spec, k)
fullPath := filepath.Join(`%{_libexecdir}`, subPath, df.ResolveName(k))
fmt.Fprintln(b, fullPath)
}
}

configKeys := dalec.SortMapKeys(w.Spec.Artifacts.ConfigFiles)
for _, c := range configKeys {
cfg := w.Spec.Artifacts.ConfigFiles[c]
Expand Down
11 changes: 11 additions & 0 deletions frontend/template.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package frontend

import "github.com/Azure/dalec"

func DefaultLibexecSubpath(s *dalec.Spec, k string) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right.
The way we handle this for the other types (like libs) is always <base>/<spec name>/<subpath>/<artifact name>

As an example, see how Libs are handled.

Also at this time I'd like to avoid abstracting the behavior here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me double check a real system for common libexec usage, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that's the way to go, and if someone needs something directly in /usr/libexec or somewhere else they can use a symlink.

Copy link
Contributor Author

@pmengelbert pmengelbert Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about when packages are renamed from upstream? For example, if we have the moby-buildx package, we want the docker-buildx binary to go into /usr/libexec/docker/cli-plugins/docker-buildx rather than /usr/libexec/moby-buildx/cli-plugins/docker-buildx. IMO we should let the user do the right thing without adding extra steps.

Copy link
Contributor Author

@pmengelbert pmengelbert Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right. The way we handle this for the other types (like libs) is always <base>/<spec name>/<subpath>/<artifact name>

That seems pretty non-standard. Pretty much everything in my /usr/libexec directory is either in /usr/libexec directly, or nested under a single subdirectory bearing the main application's name. There's only a few files nested deeper than the application's name.

If someone needs something directly in /usr/libexec or somewhere else they can use a symlink.

That would violate the standard referenced in the PR description: "Applications may use a single subdirectory under /usr/libexec". If we create a symlink, we have one application (docker) with two subdirectories under /usr/libexec (/usr/libexec/docker and /usr/libexec/moby-buildx).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a separate field to overwrite the package name is worthwhile in this case.
Or just a bool like OmitAutoPackageName

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if s.Artifacts.Libexec[k].SubPath != "" {
return s.Artifacts.Libexec[k].SubPath
}

return s.Name
}
90 changes: 90 additions & 0 deletions test/azlinux_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,96 @@ Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/boot
})
})

t.Run("test libexec file installation", func(t *testing.T) {
t.Parallel()
spec := &dalec.Spec{
Name: "libexec-test",
Version: "0.0.1",
Revision: "1",
License: "MIT",
Website: "https://github.com/azure/dalec",
Vendor: "Dalec",
Packager: "Dalec",
Description: "Should install specified data files",
Sources: map[string]dalec.Source{
"no_name_no_subpath": {
Inline: &dalec.SourceInline{
File: &dalec.SourceInlineFile{
Contents: "#!/usr/bin/env bash\necho hello world",
Permissions: 0o755,
},
},
},
"name_only": {
Inline: &dalec.SourceInline{
File: &dalec.SourceInlineFile{
Contents: "#!/usr/bin/env bash\necho hello world",
Permissions: 0o755,
},
},
},
"name_and_subpath": {
Inline: &dalec.SourceInline{
File: &dalec.SourceInlineFile{
Contents: "#!/usr/bin/env bash\necho hello world",
Permissions: 0o755,
},
},
},
"subpath_only": {
Inline: &dalec.SourceInline{
File: &dalec.SourceInlineFile{
Contents: "#!/usr/bin/env bash\necho hello world",
Permissions: 0o755,
},
},
},
},
Build: dalec.ArtifactBuild{},
Artifacts: dalec.Artifacts{
Binaries: map[string]dalec.ArtifactConfig{
"no_name_no_subpath": {},
},
Libexec: map[string]dalec.ArtifactConfig{
"no_name_no_subpath": {},
"name_only": {
Name: "name_only",
},
"name_and_subpath": {
SubPath: "custom/subpath",
Name: "custom_name",
},
"subpath_only": dalec.ArtifactConfig{
SubPath: "custom",
},
},
},
}

testEnv.RunTest(ctx, t, func(ctx context.Context, client gwclient.Client) {
req := newSolveRequest(withBuildTarget(testConfig.Target.Container), withSpec(ctx, t, spec))
res := solveT(ctx, t, client, req)

ref, err := res.SingleRef()
if err != nil {
t.Fatal(err)
}

if err := validatePathAndPermissions(ctx, ref, "/usr/libexec/libexec-test/no_name_no_subpath", 0o755); err != nil {
t.Fatal(err)
}
if err := validatePathAndPermissions(ctx, ref, "/usr/libexec/libexec-test/name_only", 0o755); err != nil {
t.Fatal(err)
}
if err := validatePathAndPermissions(ctx, ref, "/usr/libexec/custom/subpath/custom_name", 0o755); err != nil {
t.Fatal(err)
}
if err := validatePathAndPermissions(ctx, ref, "/usr/libexec/custom/subpath_only", 0o755); err != nil {
t.Fatal(err)
}
})
})

t.Run("test config files handled", func(t *testing.T) {
t.Parallel()
spec := &dalec.Spec{
Expand Down
26 changes: 26 additions & 0 deletions website/docs/artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,32 @@ artifacts:
You may use a trailing wildcard to specify multiple binaries in a directory,
though behavior may differ between different OS's/distros.

### Libexec

Libexec files are additional executable files that may be executed by one of
the main package executables. On Linux these would typically get installed into
`/usr/libexec/<package-name>`.

Files under libexec are a mapping of file path to [artifact configuration](#artifact-configuration).
If `subpath` is not supplied, it will default to the package name. The file
path is the path to a file that must be available after the build section has
finished. This path is relative to the working directory of the build phase
*before* any directory changes are made.

Example:

```yaml
name: my_package

artifacts:
# the following config will install my_bin at /usr/libexec/my package/my_bin
libexec:
src/my_bin:
```

You may use a trailing wildcard to specify multiple binaries in a directory,
though behavior may differ between different OS's/distros.

### Manpages

Manpages is short for manual pages.
Expand Down