Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a zarf package publish command #1336

Merged
merged 120 commits into from
Mar 1, 2023

Conversation

Noxsios
Copy link
Contributor

@Noxsios Noxsios commented Feb 6, 2023

Description

💲 zarf package publish

This PR adds a new zarf package publish command.

This command will publish a fully built (via zarf package create) Zarf package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's helm push command operates and has many of the same restrictions.

https://v3.helm.sh/docs/topics/registries/#the-push-subcommand

$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns

CLI Syntax restrictions:

  • The OCI URL must start with oci://

  • A basename/version cannot be provided, as it is derived from information present in the built zarf.yaml where it must be set.

  • The command may return an error, permission denied, if the repository does not exist, ie defenseunicorns/strimzi as an example on Docker Hub.

  • A zarf tools registry login must be done prior as this will inherit the credentials from docker's cred system (defaultly located at ~/.docker), or users can create a Docker compatible config.json and point its directory:

# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>

💲 zarf package deploy oci://

This PR also adds a new deploy oci:// feature.

This command uses the existing deploy system, but does some slightly different behavior from sget://. oras is used to pull the package layer by layer to the temp directory. There is no need to unarchive, as the package is already in an uncompressed state. The temp path is then used by the packager to deploy the package as though it is a local package. Usage is denoted in the following example:

$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure

💲 zarf package inspect oci://

This PR also adds a new inspect oci:// feature.

$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817

Related Issue

Relates to #1298

Fixes #381
Fixes #823

Blocked by #1331

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Checklist before merging

@netlify
Copy link

netlify bot commented Feb 6, 2023

Deploy Preview for zarf-docs canceled.

Name Link
🔨 Latest commit a1bff67
🔍 Latest deploy log https://app.netlify.com/sites/zarf-docs/deploys/63e2e43ea326890007545021

src/pkg/packager/publish.go Outdated Show resolved Hide resolved
@Noxsios Noxsios changed the base branch from main to features/oci-package February 8, 2023 04:17
@wirewc
Copy link
Contributor

wirewc commented Feb 8, 2023

For podman, make sure the following exists for the docker login to work as expected for podman.

mkdir ~/.docker/
ln -s $$XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

I don't know if we should add that into a help doc or message, or if podman should just be that much of an outlier to ignore it.

@Noxsios

This comment was marked as outdated.

@Noxsios Noxsios self-assigned this Feb 9, 2023
@Noxsios
Copy link
Contributor Author

Noxsios commented Feb 9, 2023

Awareness from @jeff-mccoy docker/docs#8230

Run: func(cmd *cobra.Command, args []string) {
pkgConfig.PublishOpts.PackagePath = choosePackage(args)

if !strings.HasPrefix(args[1], "oci://") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I would recommend using net/url for operations like this.

The following should have all the information you're looking for.

    parts, _ := url.Parse(args[1])
    parts.Scheme
    parts.Host
    parts.Path

src/pkg/packager/network.go Outdated Show resolved Hide resolved
src/pkg/packager/network.go Show resolved Hide resolved
@Noxsios Noxsios merged commit 8ac9fd2 into features/oci-package Mar 1, 2023
@Noxsios Noxsios deleted the features/oci-publish branch March 1, 2023 19:09
@Noxsios
Copy link
Contributor Author

Noxsios commented Mar 1, 2023

Going to be adding the tests in the main feature branch in the MZOCI PR: #1402

Noxsios added a commit that referenced this pull request Mar 1, 2023
## Description

### 💲 `zarf package publish`

This PR adds a new `zarf package publish` command.

This command will publish a fully built (via `zarf package create`) Zarf
package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's `helm push` command
operates and has many of the same restrictions.

<https://v3.helm.sh/docs/topics/registries/#the-push-subcommand>

```
$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns
```

#### CLI Syntax restrictions:

- The OCI URL _must_ start with `oci://`
- A basename/version _cannot_ be provided, as it is derived from
information present in the built `zarf.yaml` where it _must_ be set.

- The command may return an error, permission denied, if the repository
does not exist, ie `defenseunicorns/strimzi` as an example on Docker
Hub.
- A `zarf tools registry login` must be done prior as this will inherit
the credentials from docker's cred system (defaultly located at
`~/.docker`), or users can create a Docker compatible `config.json` and
point its directory:

```sh
# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>
```

### 💲 `zarf package deploy oci://`

This PR also adds a new `deploy oci://` feature.

This command uses the existing `deploy` system, but does some slightly
different behavior from `sget://`. oras is used to pull the package
layer by layer to the temp directory. There is no need to unarchive, as
the package is already in an uncompressed state. The temp path is then
used by the packager to deploy the package as though it is a local
package. Usage is denoted in the following example:

```
$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure
```

### 💲 `zarf package inspect oci://`

This PR also adds a new `inspect oci://` feature.

```
$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817
```

## Related Issue

Relates to #1298 

Fixes #381
Fixes #823

Blocked by #1331 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>
Co-authored-by: Wayne Starr <me@racer159.com>
Noxsios added a commit that referenced this pull request Mar 8, 2023
## Description

### 💲 `zarf package publish`

This PR adds a new `zarf package publish` command.

This command will publish a fully built (via `zarf package create`) Zarf
package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's `helm push` command
operates and has many of the same restrictions.

<https://v3.helm.sh/docs/topics/registries/#the-push-subcommand>

```
$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns
```

#### CLI Syntax restrictions:

- The OCI URL _must_ start with `oci://`
- A basename/version _cannot_ be provided, as it is derived from
information present in the built `zarf.yaml` where it _must_ be set.

- The command may return an error, permission denied, if the repository
does not exist, ie `defenseunicorns/strimzi` as an example on Docker
Hub.
- A `zarf tools registry login` must be done prior as this will inherit
the credentials from docker's cred system (defaultly located at
`~/.docker`), or users can create a Docker compatible `config.json` and
point its directory:

```sh
# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>
```

### 💲 `zarf package deploy oci://`

This PR also adds a new `deploy oci://` feature.

This command uses the existing `deploy` system, but does some slightly
different behavior from `sget://`. oras is used to pull the package
layer by layer to the temp directory. There is no need to unarchive, as
the package is already in an uncompressed state. The temp path is then
used by the packager to deploy the package as though it is a local
package. Usage is denoted in the following example:

```
$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure
```

### 💲 `zarf package inspect oci://`

This PR also adds a new `inspect oci://` feature.

```
$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817
```

## Related Issue

Relates to #1298 

Fixes #381
Fixes #823

Blocked by #1331 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>
Co-authored-by: Wayne Starr <me@racer159.com>
Noxsios added a commit that referenced this pull request Mar 8, 2023
## Description

### 💲 `zarf package publish`

This PR adds a new `zarf package publish` command.

This command will publish a fully built (via `zarf package create`) Zarf
package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's `helm push` command
operates and has many of the same restrictions.

<https://v3.helm.sh/docs/topics/registries/#the-push-subcommand>

```
$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns
```

#### CLI Syntax restrictions:

- The OCI URL _must_ start with `oci://`
- A basename/version _cannot_ be provided, as it is derived from
information present in the built `zarf.yaml` where it _must_ be set.

- The command may return an error, permission denied, if the repository
does not exist, ie `defenseunicorns/strimzi` as an example on Docker
Hub.
- A `zarf tools registry login` must be done prior as this will inherit
the credentials from docker's cred system (defaultly located at
`~/.docker`), or users can create a Docker compatible `config.json` and
point its directory:

```sh
# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>
```

### 💲 `zarf package deploy oci://`

This PR also adds a new `deploy oci://` feature.

This command uses the existing `deploy` system, but does some slightly
different behavior from `sget://`. oras is used to pull the package
layer by layer to the temp directory. There is no need to unarchive, as
the package is already in an uncompressed state. The temp path is then
used by the packager to deploy the package as though it is a local
package. Usage is denoted in the following example:

```
$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure
```

### 💲 `zarf package inspect oci://`

This PR also adds a new `inspect oci://` feature.

```
$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817
```

## Related Issue

Relates to #1298

Fixes #381
Fixes #823

Blocked by #1331

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>
Co-authored-by: Wayne Starr <me@racer159.com>
Signed-off-by: razzle <harry@razzle.cloud>
Racer159 added a commit that referenced this pull request Mar 8, 2023
## Description

### 💲 `zarf package publish`

This PR adds a new `zarf package publish` command.

This command will publish a fully built (via `zarf package create`) Zarf
package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's `helm push` command
operates and has many of the same restrictions.

<https://v3.helm.sh/docs/topics/registries/#the-push-subcommand>

```
$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns
```

#### CLI Syntax restrictions:

- The OCI URL _must_ start with `oci://`
- A basename/version _cannot_ be provided, as it is derived from
information present in the built `zarf.yaml` where it _must_ be set.

- The command may return an error, permission denied, if the repository
does not exist, ie `defenseunicorns/strimzi` as an example on Docker
Hub.
- A `zarf tools registry login` must be done prior as this will inherit
the credentials from docker's cred system (defaultly located at
`~/.docker`), or users can create a Docker compatible `config.json` and
point its directory:

```sh
# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>
```

### 💲 `zarf package deploy oci://`

This PR also adds a new `deploy oci://` feature.

This command uses the existing `deploy` system, but does some slightly
different behavior from `sget://`. oras is used to pull the package
layer by layer to the temp directory. There is no need to unarchive, as
the package is already in an uncompressed state. The temp path is then
used by the packager to deploy the package as though it is a local
package. Usage is denoted in the following example:

```
$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure
```

### 💲 `zarf package inspect oci://`

This PR also adds a new `inspect oci://` feature.

```
$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817
```

## Related Issue

Relates to #1298 

Fixes #381
Fixes #823

Blocked by #1331 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>
Co-authored-by: Wayne Starr <me@racer159.com>
Racer159 added a commit that referenced this pull request Mar 8, 2023
## Description

### 💲 `zarf package publish`

This PR adds a new `zarf package publish` command.

This command will publish a fully built (via `zarf package create`) Zarf
package to an OCI compliant registry using the oras library.

The command's usage is based off of how Helm's `helm push` command
operates and has many of the same restrictions.

<https://v3.helm.sh/docs/topics/registries/#the-push-subcommand>

```
$ zarf package publish [PATH_TO_TAR] [REGISTRY_URL]

$ zarf package publish zarf-package-strimzi.tar oci://localhost:666/defenseunicorns
```

#### CLI Syntax restrictions:

- The OCI URL _must_ start with `oci://`
- A basename/version _cannot_ be provided, as it is derived from
information present in the built `zarf.yaml` where it _must_ be set.

- The command may return an error, permission denied, if the repository
does not exist, ie `defenseunicorns/strimzi` as an example on Docker
Hub.
- A `zarf tools registry login` must be done prior as this will inherit
the credentials from docker's cred system (defaultly located at
`~/.docker`), or users can create a Docker compatible `config.json` and
point its directory:

```sh
# example symlinking podman
$ mkdir ~/.docker/
$ ln -s $XDG_RUNTIME_DIR/containers/auth.json ~/.docker/config.json

$ zarf package publish <...>
```

### 💲 `zarf package deploy oci://`

This PR also adds a new `deploy oci://` feature.

This command uses the existing `deploy` system, but does some slightly
different behavior from `sget://`. oras is used to pull the package
layer by layer to the temp directory. There is no need to unarchive, as
the package is already in an uncompressed state. The temp path is then
used by the packager to deploy the package as though it is a local
package. Usage is denoted in the following example:

```
$ zarf package deploy oci://REGISTRY/NAMESPACE/NAME:VERSION

$ zarf package deploy oci://docker.io/defenseunicorns/strimzi:v0.24.0-arm64

$ zarf package deploy oci://localhost:666/strimzi:v0.24.0-arm64 --insecure
```

### 💲 `zarf package inspect oci://`

This PR also adds a new `inspect oci://` feature.

```
$ zarf package inspect oci://docker.io/defenseunicorns/strimzi

tags:
- v0.23.5-14-arm64
latest:
  tag: v0.23.5-14-arm64
  descriptor:
    mediaType: application/vnd.oci.image.manifest.v1+json
    digest: sha256:340f489a105e476f846203e6844b2738f2924fc608522711761a21b176d6b67f
    size: 41817
```

## Related Issue

Relates to #1298 

Fixes #381
Fixes #823

Blocked by #1331 

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/zarf/blob/main/CONTRIBUTING.md#developer-workflow)
followed

---------

Co-authored-by: Wayne Starr <Racer159@users.noreply.github.com>
Co-authored-by: Wayne Starr <me@racer159.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-tests PR Label - Tests required to merge oci
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants