diff --git a/MAINTAINERS b/MAINTAINERS index c47ce4c..99ede26 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2,4 +2,4 @@ + Derek McGowan (@dmcgowan) + Mike Brown (@mikebrow) + Stephen Day (@stevvooe) -+ Joey Schorr (@josephschorr) ++ Joey Schorr (@josephschorr) diff --git a/README.md b/README.md index f7fe90e..9bf0eb8 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,17 @@ -# OCI Artifacts +# OCI Artifacts (Experimental) with OCI Artifact Reference Support -## Artifact Guidance Documents +This is an experimental branch of oci artifacts, validating the [oci.artifact.manifest spec][oci-artifact-manifest-spec] support of reference types. Reference types are required to meet the [Notary v2 Requirements][nv2-requirements] for not changing the target digest or tag, and the [Notary v2 Scenarios][nv2-scenarios] for content movement within and across registry implementations. Reference types enable a wider range of scenarios, including secure supply chain artifacts that may be represented as a graph as they move across environments. -1. [Artifact Author Guidance](./artifact-authors.md) +![](media/net-monitor-graph.svg) -## Supporting Documents +## Table of Contents: -- [Term Definitions](./definitions-terms.md) +- [OCI Artifact Manifest Overview][oci-artifact-manifest] +- [OCI Artifact Reference Type Manifest Spec](./artifact-reftype-spec.md) +- [ORAS experimental support for oci.artifact.manifest references][oras-artifacts] to `push`, `discover`, `pull` referenced artifact types. -## Project Introduction and Scope - -Container registries, implementing the [distribution-spec][distribution-spec], provide reliable, highly scalable, secured storage services for container images. Customers either use a cloud provider implementation, vendor implementations, or instance the open source implementation of [distribution][distribution]. They configure security and networking to assure the images in the registry are locked down and accessible by the resources required. Cloud providers and vendors often provide additional values atop their registry implementations from security to productivity features. - -Applications and services typically require additional artifacts to deploy and manage, including [helm](https://helm.sh) for deployment and [Open Policy Agent (OPA)](https://github.com/open-policy-agent/opa/issues/1413) for policy enforcement. - -Utilizing the [manifest][image-manifest] and [index][image-index] definitions, new artifacts, such as the [Singularity project][singularity], can be stored and served using the [distribution-spec][distribution-spec]. - -This repository provides a reference for artifact authors and registry implementors for supporting new artifact types with the existing implementations of distribution. -More particularly this repository has been tasked by the [OCI TOB](https://github.com/opencontainers/tob/blob/master/proposals/artifacts.md) to serve 3 primary goals: - -1. **artifact authors** - [guidance for authoring new artifact types.][artifact-authors] Including a clearing house for well known artifact types. -1. **registry operators and vendors** - guidance for how operators and vendors can support new artifact types, including how they can opt-in or out of well known artifact types. Registry operators that already implement `media-type` filtering will not have to change. The artifact repo will provide context on how new `media-type`s can be used, and how `media-type`s can be associated with a type of artifact. -1. **clearing house for well known artifacts** - artifact authors can submit their artifact definitions, providing registry operators a list by which they can easily support. - -By providing an OCI artifact definition, the community can continue to innovate, focusing on new artifact types without having to build yet another storage solution (YASS). - -## Project Governance and License - -- [Artifact Authors- How To][artifact-authors] -- [The Apache License, Version 2.0](LICENSE) -- [Maintainers](MAINTAINERS) -- [Maintainer guidelines](MAINTAINERS_GUIDE.md) -- [Contributor guidelines](CONTRIBUTING.md) -- [Project governance](GOVERNANCE.md) -- [Release procedures](RELEASES.md) - -## Code of Conduct - -This project incorporates (by reference) the OCI [Code of Conduct][code-of-conduct]. - -## Governance and Releases - -This project incorporates the Governance and Releases processes from the OCI project template: https://github.com/opencontainers/project-template. - -## Project Communications - -This project would continue to use existing channels in use by the [OCI developer community for communication](https://github.com/opencontainers/org#communications) - -### Versioning / Roadmap - -Artifacts will reference specific [distribution][distribution-spec], [index][image-index] and [manifest][image-manifest] versions in its examples, identifying any dependencies required. - -## Frequently Asked Questions (FAQ) - -**Q: Does this change the OCI Charter or Scope Table?** - -A: No. Artifacts are a prescriptive means of storing [index][image-index] and [manifest][image-manifest] within [distribution][distribution-spec] implementations. - -[artifact-authors]: ./artifact-authors.md -[code-of-conduct]: https://github.com/opencontainers/org/blob/master/CODE_OF_CONDUCT.md -[distribution]: https://github.com/docker/distribution -[distribution-spec]: https://github.com/opencontainers/distribution-spec/ -[image-index]: https://github.com/opencontainers/image-spec/blob/master/image-index.md -[image-manifest]: https://github.com/opencontainers/image-spec/blob/master/manifest.md -[singularity]: https://github.com/sylabs/singularity +[oci-artifact-manifest]: ./artifact-manifest.md +[oci-artifact-manifest-spec]: ./artifact-reftype-spec.md +[nv2-requirements]: https://github.com/notaryproject/notaryproject/blob/main/requirements.md +[nv2-scenarios]: https://github.com/notaryproject/notaryproject/blob/main/scenarios.md +[oras-artifacts]: https://github.com/deislabs/oras/blob/prototype-2/docs/artifact-manifest.md \ No newline at end of file diff --git a/artifact-authors.md b/artifact-authors.md index c3b043d..cd7a7c4 100644 --- a/artifact-authors.md +++ b/artifact-authors.md @@ -53,7 +53,7 @@ The manifest `config.mediaType` is the equivalent of a file extension, enabling |Icon|Artifact|`config.mediaType`| |-|-|-| ||[OCI Image][image-spec]|`application/vnd.oci.image.config.v1+json`| -||[Helm Chart](https://helm.sh)|`application/vnd.cncf.helm.chart.config.v1+json`| +||[Helm Chart](https://helm.sh)|`application/vnd.cncf.helm.chart.config.v1+json`| ||[Singularity][singularity], by [Sylabs][sylabs]|`application/vnd.sylabs.sif.config.v1+json`| ## Defining a Unique Artifact Type diff --git a/artifact-manifest-spec.md b/artifact-manifest-spec.md new file mode 100644 index 0000000..f39d70d --- /dev/null +++ b/artifact-manifest-spec.md @@ -0,0 +1,76 @@ +# OCI Artifact Manifest Spec (Phase-1 Reference Types) +The OCI artifact manifest generalizes the use cases of [OCI image manifest][oci-image-manifest-spec] by removing constraints defined on the image-manifest such as a required `config` object and required & ordinal `layers`. It then adds a `subjectManifest` property supporting reference types. The addition of a new manifest does not change, nor impact the `image.manifest`. It provides a means to define a wide range of artifacts, including a chain of related artifacts enabling SBoMs, on-demand loading, signatures and metadata that can be related to an `image.manifest` or `image.index`. By defining a new manifest, registries and clients opt-into new capabilities, without breaking existing registry and client behavior or setting expectations for scenarios to function when the client and/or registry doesn't yet implement the new capabilities. + +To enable a fall 2021 focus on supply chain security, **Phase 1** will narrowly focus on Reference Type support, giving time for further generalization with less time constraints. + +For usage and scenarios, see [artifact-manifest.md](./artifact-manifest.md) + +## Example OCI Artifact Manifests + +The following are Phase 1 examples: + +- [`net-monitor:v1` oci container image](./artifact-manifest/net-monitor-oci-image.json) +- [`net-monitor:v1` notary v2 signature](./artifact-manifest/net-monitor-image-signature.json) +- [`net-monitor:v1` sample sbom](./artifact-manifest/net-monitor-image-sbom.json) +- [`net-monitor:v1` nydus image with on-demand loading](./artifact-manifest/net-monitor-image-nydus-ondemand-loading.json) + +## OCI Artifact Manifest Properties + +For **Phase 1**, an artifact manifest provides an optional collection of blobs and a reference to the manifest of another artifact. + +- **`schemaVersion`** *int* + + This REQUIRED property specifies the artifact manifest schema version. + For this version of the specification, this MUST be `3`. The value of this field WILL change as the manifest schema evolves. Minor version changes to the `oci.artifact.manifest` spec MUST be additive, while major version changes MAY be breaking. Artifact clients MUST implement version checking to allow for future, yet unknown changes. Artifact clients MUST ignore additive properties to minor versions. Artifact clients MAY support major changes, with no guarantee major changes MAY impose breaking changing behaviors. Artifact authors MAY support new and older schemaVersions to provide the best user experience. + +- **`mediaType`** *string* + + This field contains the `mediaType` of this document, differentiating from [image-manifest][oci-image-manifest-spec] and [oci-image-index]. The mediaType for this manifest type MUST be `application/vnd.oci.artifact.manifest.v1+json`, where the version WILL change to reflect newer versions. Artifact authors SHOULD support multiple `mediaType` versions to provide the best user experience for their artifact type. + +- **`artifactType`** *string* + + Phase 1 of the OCI Artifact spec will support reference types to existing [OCI Artifacts][oci-artifacts]. The REQUIRED `artifactType` is unique value, as registered with iana.org. See [registering unique types.][registering-iana]. The `artifactType` is equivalent to OCI Artifacts that used the `manifest.config.mediaType` to differentiate the type of artifact. Artifact authors that implement `oci.artifact.manifest` use `artifactType` to differentiate the type of artifact. example:(`example.sbom` from `cncf.notary`). + +- **`blobs`** *array of objects* + + An OPTIONAL collection of 0 or more blobs. The blobs array is analogous to [oci.image.manifest layers][oci-image-manifest-spec-layers], however unlike [image-manifest][oci-image-manifest-spec], the ordering of blobs is specific to the artifact type. Some artifacts may choose an overlay of files, while other artifact types may store indepdent collections of files. + + - Each item in the array MUST be a [descriptor][descriptor], and MUST NOT refer to another `manifest` providing dependency closure. + - The max number of blobs is not defined, but MAY be limited by [distribution-spec][oci-distribution-spec] implementations. + - An encountered `descriptor.mediaType` that is unknown to the implementation MUST be ignored. + +- **`subjectManifest`** *descriptor* + + An OPTIONAL reference to any existing manifest within the repository. When specified, the artifact is said to be dependent upon the referenced `subjectManifest`. + - The item MUST be a [descriptor][descriptor] representing a manifest. Descriptors to blobs are not supported. The registry MUST return a `400` response code when `subjectManifest` is not found in the same repository, and not a manifest. + +- **`annotations`** *string-string map* + + This OPTIONAL property contains arbitrary metadata for the image manifest. + This OPTIONAL property MUST use the [annotation rules](annotations.md#rules). + + See [Pre-Defined Annotation Keys][annotations] + +## Push Validation + +Following the [distribution-spec push api](https://github.com/opencontainers/distribution-spec/blob/main/spec.md#push), all `blobs` *and* the `subjectManifest` descriptors SHOULD exist when pushed to a distribution instance. + +## Lifecycle Management + +For Phase 1, artifact types will be limited to reference types. A reference type is an artifact that doesn't have a lifecycle unto itself. A container image is said to have an independent lifecycle. A reference type, such as an SBoM or signature have a lifecycle tied to the `subjectManifest`. When the `subjectManifest` is deleted or marked for garbage collection, the defined artifact is subject to deletion as well. A distribution instance SHOULD delete, (refCount -1) the artifact when the `subjectManifest` is deleted. + +### Tagged `referenceTypes` + +As signatures and SBoMs are not considered independent artifact types, they SHOULD NOT have a tag, simplifying the lifecycle management. As the `subjectManifest` is marked for deletion (refCount=0), the `referenctType` is also marked for deletion (refCount -1). However, these artifacts MAY have tags as future versions of the artifact manifest MAY support independent types. + +[oci-artifacts]: https://github.com/opencontainers/artifacts +[oci-config]: https://github.com/opencontainers/image-spec/blob/master/config.md +[oci-image-manifest-spec]: https://github.com/opencontainers/image-spec/blob/master/manifest.md +[oci-image-manifest-spec-layers]: https://github.com/opencontainers/image-spec/blob/master/manifest.md#image-manifest-property-descriptions +[oci-image-index]: https://github.com/opencontainers/image-spec/blob/master/image-index.md +[oci-distribution-spec]: https://github.com/opencontainers/distribution-spec +[media-type]: https://github.com/opencontainers/image-spec/blob/master/media-types.md +[artifact-type]: https://github.com/opencontainers/artifacts/blob/master/artifact-authors.md#defining-a-unique-artifact-type +[registering-iana]: ./artifact-authors.md#registering-unique-types-with-iana +[descriptor]: https://github.com/opencontainers/image-spec/blob/master/descriptor.md +[annotations]: https://github.com/opencontainers/image-spec/blob/master/annotations.md \ No newline at end of file diff --git a/artifact-manifest.md b/artifact-manifest.md new file mode 100644 index 0000000..f1b468c --- /dev/null +++ b/artifact-manifest.md @@ -0,0 +1,382 @@ +# OCI Artifact Manifest + +The OCI artifact manifest generalizes the use cases of [OCI image manifest][oci-image-manifest-spec] by removing constraints defined on the image-manifest such as a required `config` object and required ordinal `layers` collection. The `subjectManifest` property provides a means to define a wide range of artifacts, including a chain of related artifacts enabling SBoMs, on-demand loading, signatures and metadata that can be related to an `image.manifest` or `image.index`. The addition of a new manifest does not change, nor impact the `image.manifest`. By defining a new manifest, registries and clients opt-into new capabilities, without breaking existing registry and client behavior or setting expectations for scenarios to function when the client and/or registry doesn't yet implement the new capabilities. + +# Phased Implementations Through Experimental Releases + +To meet pressing timelines for delivering Secure Supply Chain artifacts in the fall of 2021, the spec will be split into two phases: + +### Phase 1 - Reference Types + +Phase 1 is a time focused release, prioritizing a subset of capabilities needed to support reference types enabling signing and sbom scenarios. + +Reference types require: +- **Manifest reference**: a means to add content to a registry, referencing existing, unchanged, content. +- **Reference Discovery API**, where the consumer finds referenced artifacts by querying what artifacts are related to a subject artifact. + For example, what signatures or SBoMs are related to the `net-monitor:v1` container image. See the [manifest-referrers api](./manifest-referrers-api.md) for details. +- **Lifecycle management**: as content is added to a registry, how is its lifecycle handled? Can a user can find and delete reference types, and how would a registry garbage collect unreferenced content. + As registries implement the [distribution-spec][oci-distribution-spec], content may be stored indefinitely. To assure registries MAY implement garbage collection, a manifest is used to identify the intent of the content. See [Lifecycle Management][lifecycle-management] for details. The spec doesn't dictate how an lifecycle management must be implemented, rather focuses on a set of consistent expectations for users to have when working across different implementations. + +To separate the reference type deliverables for the fall of 2021 from future work, a `application/vnd.oci.artifact.manifest.v1+json` is provided. +For spec details on Phase 1, see [artifact-spec.md](./artifact-spec.md) + +### Phase 2 - Artifact Versioning Support + +Phase 2 will work to enable the full range of scenarios outlined in [WIP generic object spec #37](https://github.com/opencontainers/artifacts/pull/37). + +By splitting out **Phase 1** from **Phase 2**, focus is placed upon a subset of capabilities in a short time, while providing time to review and evolve the larger scenarios. + +The goal is to migrate phase 1 implementations to phase 2 (derivative of PR #37). It is not the intent to have dozens of manifest types. Phase 1 is a point in time release to meet pressing security requirements for supply chain artifacts, while building knowledge to feed into phase 2. + +### Migrating from Phase 1 to Phase 2 + +Registries that implement phase 1 will likely focus on implementing reverse indexes, supporting the referrers api and lifecycle management to assure untagged reference types aren't automatically deleted. While the manifest format will change, the underlying capabilities for reverse indexes and lifecycle management will be maintained. + +All `oci.artifact.*` based content will be named and versioned, enabling distribution instances and distribution clients to determine the content and how it may be processed. + +## Reference Types + +There are a new set of scenarios requiring the ability to reference existing artifacts, including the ability to additively sign content or add a SBoM. The addition of a [`subjectManifest`][subjectManifest] property supports linking artifacts through a reference from one artifact manifest to another artifact manifest. By storing these as separate, but linked artifacts, the existing OCI Image tool chain remains unchanged. Tooling that opts into understanding these reference types (eg. SBoM, Notary v2 signatures and Nydus image with on-demand loading) can find the referenced artifacts without changing the existing image tool chains. + +### Example OCI Artifact Manifests + +- [`net-monitor:v1` oci container image](./artifact-manifest/net-monitor-oci-image.json) +- [`net-monitor:v1` notary v2 signature](./artifact-manifest/net-monitor-image-signature.json) +- [`net-monitor:v1` sample sbom](./artifact-manifest/net-monitor-image-sbom.json) +- [`net-monitor:v1` nydus image with on-demand loading](./artifact-manifest/net-monitor-image-nydus-ondemand-loading.json) + +## Reference Type Requirements + +Reference type persistance & discovery supports the following requirements: + +- Maintain the original artifact digest and collection of associated tags, supporting existing dev through deployment workflows. +- Multiple references per artifact, enabling multiple signatures, SBoMs, images with on-demand loading and other reference types. +- Avoiding the need to tag a reference type to assure its not garbage collected as an untagged manifest. +- Native persistence within an OCI Artifact enabled, distribution spec based registry. +- Copying the graph of references within and across OCI Artifact enabled, distribution spec based registries, enabling an image, its signatures, SBoMs and images with on-demand loading to be copied as a group. + +To support the above requirements, reference types (eg signatures, SBoMs, images with on-demand loading) are stored as individual, untagged [OCI Artifacts][oci-artifacts]. They are maintained as any other artifact in a registry, supporting standard operations such as listing, deleting, garbage collection and any other content addressable operations within a registry. Untagged artifacts are considered not ready for garbage collection if they have a reference to an existing artifact. See [Lifecycle Management][lifecycle-management] for spec details. + +### Example Image + +The `net-monitor:v1` image contains a config and a collection of layers, as defined by the [oci.image.manifest spec][oci-image-manifest-spec]. + +![OCI Image](./media/net-monitor-image.svg) + +The `net-monitor:v1` image is persisted as an `oci.image.manifest`, with a unique digest. + + +- **repository**: `net-monitor` +- **digest**: `sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333` +- **tag**: `:v1` +> **NOTE:** There is no change to the existing image-spec 1.0 format. This example highlights what already exists. + ```json + { + "schemaVersion": 2, + "config": { + "mediaType": "application/vnd.oci.image.config.v1+json", + "digest": "sha256:e752324f6804d5d0b2c098f84507d095a8fd0031cf06cdb3c7ad1625dcd1b399", + "size": 7097 + }, + "layers": [ + { + "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", + "digest": "sha256:83c5cfdaa5385ea6fc4d31e724fd4dc5d74de847a7bdd968555b8f2c558dac0e", + "size": 25851449 + }, + { + "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", + "digest": "sha256:7445693bd43e8246a8c166233392b33143f7f5e396c480f74538e5738fb6bd6e", + "size": 226 + } + ] + } + ``` + +### Notary v2 Signatures and SBoM Persistance + +Following the [oci.artifact.manifest spec][oci-artifact-manifest-spec], reference type artifacts are pushed with an `manifest.artifactType`, and a `subjectManifest` for the artifact referenced. + +A signature, or an SBoM, would be persisted with the content persisted in the `[blobs]` collection, and a `subjectManifest` referencing the `net-monitor:v1` image (by digest). + +![Notary v2 signature](./media/notaryv2-signature.svg) + +**An Artifact Manifest, capturing the Notary v2 signature of the `net-monitor:v1` image:** + +- **repository**: `net-monitor` +- **digest**: `sha256:8ac803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c222` +- **tag**: _-none-_ + ```json + { + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "cncf.notary.v2-rc1", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333", + "size": 16724 + }, + "annotations": { + "org.cncf.notary.v2.signature.subject": "wabbit-networks.io" + } + } + ``` + +The same `net-monitor:v1` image may have an associated SBoM. The SBoM content would be persisted as one or more `[blobs]` with a `subjectManifest` referencing the `net-monitor:v1` image (by digest). + +![Sample SBOM](./media/net-monitor-sbom.svg) + +- **repository**: `net-monitor` +- **digest**: `sha256:7a781a3930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c1a` +- **tag**: _-none-_ + ```json + { + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "example.sbom.v0", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333", + "size": 16724 + }, + "annotations": { + "example.sbom.author": "wabbit-networks.io" + } + } + ``` + +The `net-monitor:v1` SBoM will also be signed, providing yet another leaf node. + +![](media/net-monitor-sbom-signature.svg) + +- **repository**: `net-monitor` +- **digest**: `sha256:ea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae14` +- **tag**: _-none-_ + ```json + { + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "cncf.notary.v2-rc1", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "digest": "sha256:7a781a3930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c1a", + "size": 16724 + }, + "annotations": { + "org.cncf.notary.v2.signature.subject": "wabbit-networks.io" + } + } +Once all artifacts are submitted, the registry would represent a graph of the `net-monitor:v1` image, including a signature, an SBoM, and a signature on the SBoM. + +![net-monitor image with an sbom & signatures](media/net-monitor-graph.svg) + +The Notary v2 signature and SBoM reference the `net-monitor:v1` image (as a digest) through the `subjectManifest` property. The `net-monitor:v1` image is represented as an oci-image, and requires no changes to its manifest to support the enhancements. The directionality of the `subjectManifest` reference enables links to existing content, without changing the existing content. + +### Deletion and Ref Counting + +The `subjectManifest` reference is a hard reference. Just as the layers of an OCI Image are deleted (*ref-counted -1*), any artifacts with a `subjectManifest` referring to the target manifest SHOULD be deleted (*ref-counted -1*). See [Lifecycle Management Spec][lifecycle-management] for details. + +## Artifact Manifest Scenarios + +The main scenarios include: + +1. [Content Discovery](#content-discovery) +1. [Content Promotion Within and Across Registries](#content-promotion-within-and-across-registries) +1. [Lifetime management](#lifetime-management), including deletion of artifacts and their linked manifests. + +### Content Discovery + +Registries support a list of content within a repository. A container image, multi-arch container image, Helm Chart, WASM and other OCI Artifact types can be listed based on their `manifest.config.mediaType` + +![flat listing of OCI artifacts](media/repo-listing-flat.svg) + +In the above example, all the artifacts are displayed without relation to each other. The image signature, SBoM, SBoM signature, Helm signature are listed with digests as they have no individual identity. However, the registry has no knowledge these artifacts are references of the image, SBoM or Helm chart. + +![flat listing of OCI artifacts](media/repo-listing-attributed.svg) + +In the above example, the Notary v2 signature, an SBoM and a collection of attributes are displayed as associated with their target artifact. The references can be collapsed as the `oci.artifact.manifest` provides the reference information. + +![expanded listing of OCI artifacts](media/repo-listing-attributed-expanded.svg) + +Using the references, the graph can be expanded providing additional information on each referenced artifact. + +See [`/referrers`][referrers-api] API for more information on listing referenced content. + +## Content Promotion Within and Across Registries + +Artifacts are promoted within and across different registries. They may be promoted from dev, through test, to production. They may continue movement to a public distribution point or deployment to an air-gapped environment. As artifacts are promoted, content related to that artifact must be capable of moving with the artifact. The OCI artifact manifest provides manifest references enabling discovery and promotion. + +### Example of Content Movement Within and Across Registries + +**Example**: Content promoted to environment specific repositories, within the same registry: + +```bash +registry.acme-rockets.io/ + dev\ + net-monitor:v1 + net-monitor:v2 + net-monitor:v3 + net-monitor-chart:v1 + net-monitor-chart:v2 + net-monitor-chart:v3 + staging/ + net-monitor:v2 + net-monitor:v3 + net-monitor-chart:v2 + net-monitor-chart:v3 + prod/ + net-monitor:v2 + net-monitor-chart:v2 +``` + +**Example**: Content promoted across different registries: + +```bash +dev-registry.acme-rockets.io/ + net-monitor:v1 + net-monitor:v2 + net-monitor:v3 + net-monitor-chart:v1 + net-monitor-chart:v2 + net-monitor-chart:v3 +``` + +is promoted to: + +```bash +prod-registry.acme-rockets.io/ + net-monitor:v2 + net-monitor-chart:v2 +``` + +**Example**: Content published for public consumption: + +```bash +products.wabbit-networks.io/ + net-monitor:v1 + charts/net-monitor:v1 +``` + +### Copying an OCI Image + +![net-monitor image copy](media/net-monitor-copy.svg) + +As an example, copying an image from a public registry to a private registry would involve `docker pull`, `docker tag` and `docker push` + +```bash +docker pull net-monitor:v1 +docker tag net-monitor:v1 registry.acme-rockets.io/base-artifacts/net-monitor:v1 +docker push registry.acme-rockets.io/base-artifacts/net-monitor:v1 +``` + +The above commands account for the image manifest and the associated layers. Note the directionality of the manifest `-->` config and layers references. A manifest declares the config and layers that must be accounted for before a manifest may be considered valid within a registry. + +### Copying an OCI Image with References + +![net-monitor image copy, with signatures](./media/net-monitor-with-sigs-copy.svg) + +Notary v2 signatures and a Notary v2 signed SBoM have been added to the `net-monitor:v1` image. Note the directionality of the SBoM and Notary v2 signature references. The Notary v2 signature and SBoM `-->` reference the `net-monitor:v1` image. From a user experience perspective, copying an image from a public registry to a private registry should copy the signatures and SBoM alongside the artifact they've signed. The OCI artifact manifest provides the information needed for a registry to index references from either direction. + +### OCI-Registry CLI + +To copy the above image and the associated signatures, an `oci-reg` cli is used for illustrative purposes. The `oci-reg` cli is an example of tools that _could_ be built by the community, as they would work within and across different OCI conformant registry implementations. + +The following command would copy the `net-monitor:v1` image from docker hub to the acme-rockets registry. The CLI _could_ be run within the source or target cloud eliminating the download/upload network hops. + +```bash +oci-reg copy \ + --source docker.io/wabbitnetworks/net-monitor \ + --target registry.acme-rockets.io/base-artifacts/net-monitor:v1 +``` + +The `oci-reg copy` command would: + +- assure the manifest and layer/blob digests remain the same +- copy any artifacts that are dependent on the source artifact-manifest, persisting them in the target registry. These _could_ include Notary v2 signatures, SBoMs, GPL source or other referenced artifacts. + +Since the artifacts are individually stored in a registry, shallow copies can be made: + +**Example**: Optional parameters to include|exclude reference types: + +```bash +oci-reg copy \ + --source docker.io/wabbitnetworks/net-monitor \ + --target registry.acme-rockets.io/base-artifacts/net-monitor:v1 \ + --copy-references disabled +``` + +As the referenced types are defined by the `manifest.subjectManifest`, copying specific content may be enabled: + +**Example**: Filter the types of enhancements: + +```bash +oci-reg copy \ + --source docker.io/wabbitnetworks/net-monitor \ + --target registry.acme-rockets.io/base-artifacts/net-monitor:v1 \ + --include-references org.cncf.notary.v2 +``` + +### Lifetime Management + +Using the OCI artifact manifest, OCI distribution-spec APIs can provide standard delete operations, including options for deleting referenced artifacts. The registry, nor the `oci-reg` cli would need to know about specific artifact type implementations. + +**Example**: Deleting images, with their Notary v2 and SBoM references: + +```bash +oci-reg delete registry.acme-rockets.io/net-monitor:v1 +``` + +**Example**: Deleting artifact references: + +```bash +oci-reg delete-references registry.acme-rockets.io/net-monitor:v1 +``` + +**Example**: Deleting specific artifact reference types: + +```bash +oci-reg delete-references \ + --referenceType org.cncf.notary.v2 \ + registry.acme-rockets.io/net-monitor:v1 +``` + +**Example**: Deleting a specific artifact reference: + +```bash +oci-reg delete registry.acme-rockets.io/net-monitor@sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7 +``` + +## Further reading + +- [oci.artifact.manifest spec][oci-artifact-manifest-spec] for more info on the manifest +- [Referrers API][referrers-api] for more information on listing references + +[lifecycle-management]: ./artifact-reftype-spec.md#lifecycle-management +[oci-image-manifest-spec]: https://github.com/opencontainers/image-spec/blob/master/manifest.md +[oci-artifacts]: https://github.com/opencontainers/artifacts +[oci-artifact-manifest-spec]: ./artifact-manifest-spec.md +[oci-image-index]: https://github.com/opencontainers/image-spec/blob/master/image-index.md +[oci-distribution-spec]: https://github.com/opencontainers/distribution-spec +[referrers-api]: ./manifest-referrers-api.md +[subjectManifest]: ./artifact-reftype-spec.md#oci-artifact-manifest-properties \ No newline at end of file diff --git a/artifact-manifest/net-monitor-image-nydus-ondemand-loading.json b/artifact-manifest/net-monitor-image-nydus-ondemand-loading.json new file mode 100644 index 0000000..4066e1f --- /dev/null +++ b/artifact-manifest/net-monitor-image-nydus-ondemand-loading.json @@ -0,0 +1,30 @@ +{ + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "cncf.nydus.v1-rc1", + "blobs": [ + { + "mediaType": "application/vnd.cncf.nydus.bootstrap.v1.tar+gzip", + "digest": "sha256:f6bb0822fe567c98959bb87aa316a565eb1ae059c46fa8bba65b573b4489b44d", + "size": 32654 + }, + { + "mediaType": "application/vnd.cncf.nydus.blob.v1", + "digest": "sha256:d35808e58856ef91d07dedf94b93301b6efdfcc831477d3b1bb37e6c3e19cce6", + "size": 25851449 + }, + { + "mediaType": "application/vnd.cncf.nydus.blob.v1", + "digest": "sha256:dbad66bcfe29ef383157a3e122acbd08cd2ebd40f5658afa2ae62c52ffe26e9f", + "size": 226 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333", + "size": 16724 + }, + "annotations": { + "org.cncf.nydus.v1.author": "wabbit-networks.io" + } +} diff --git a/artifact-manifest/net-monitor-image-sbom.json b/artifact-manifest/net-monitor-image-sbom.json new file mode 100644 index 0000000..c811307 --- /dev/null +++ b/artifact-manifest/net-monitor-image-sbom.json @@ -0,0 +1,20 @@ +{ + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "example.sbom.v0", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333", + "size": 16724 + }, + "annotations": { + "example.sbom.author": "wabbit-networks.io" + } +} diff --git a/artifact-manifest/net-monitor-image-signature.json b/artifact-manifest/net-monitor-image-signature.json new file mode 100644 index 0000000..5c0c1c8 --- /dev/null +++ b/artifact-manifest/net-monitor-image-signature.json @@ -0,0 +1,20 @@ +{ + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "artifactType": "cncf.notary.v2-rc1", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:73c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333", + "size": 16724 + }, + "annotations": { + "org.cncf.notary.v2.signature.subject": "wabbit-networks.io" + } +} diff --git a/artifact-manifest/net-monitor-oci-image.json b/artifact-manifest/net-monitor-oci-image.json new file mode 100644 index 0000000..409d859 --- /dev/null +++ b/artifact-manifest/net-monitor-oci-image.json @@ -0,0 +1,20 @@ +{ + "schemaVersion": 2, + "config": { + "mediaType": "application/vnd.oci.image.config.v1+json", + "digest": "sha256:e752324f6804d5d0b2c098f84507d095a8fd0031cf06cdb3c7ad1625dcd1b399", + "size": 7097 + }, + "layers": [ + { + "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", + "digest": "sha256:83c5cfdaa5385ea6fc4d31e724fd4dc5d74de847a7bdd968555b8f2c558dac0e", + "size": 25851449 + }, + { + "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip", + "digest": "sha256:7445693bd43e8246a8c166233392b33143f7f5e396c480f74538e5738fb6bd6e", + "size": 226 + } + ] +} \ No newline at end of file diff --git a/considerations.md b/considerations.md new file mode 100644 index 0000000..77c3ef4 --- /dev/null +++ b/considerations.md @@ -0,0 +1,136 @@ +# Extensibility + +Implementations that are reading/processing manifests MUST NOT generate an error if they encounter an unknown property. +Instead they MUST ignore unknown properties. + +# Canonicalization + +* OCI Artifacts are [content-addressable](https://en.wikipedia.org/wiki/Content-addressable_storage). See [descriptors](descriptor.md) for more. +* One benefit of content-addressable storage is easy deduplication. +* Many artifacts might depend on a particular blob, but there will only be one blob in the store. +* With a different serialization, that same semantic blob would have a different hash, and if both versions of the blob are referenced there will be two blobs with the same semantic content. +* To allow efficient storage, implementations serializing content for blobs SHOULD use a canonical serialization. +* This increases the chance that different implementations can push the same semantic content to the store without creating redundant blobs. + +## JSON + +[JSON][] content SHOULD be serialized as [canonical JSON][canonical-json]. + +Implementations: + +* [Go][]: [github.com/docker/go][], which claims to implement [canonical JSON][canonical-json] except for Unicode normalization. + +## EBNF + +For field formats described in this specification, we use a limited subset of [Extended Backus-Naur Form][ebnf], similar to that used by the [XML specification][xmlebnf]. +Grammars present in the OCI specification are regular and can be converted to a single regular expressions. +However, regular expressions are avoided to limit ambiguity between regular expression syntax. +By defining a subset of EBNF used here, the possibility of variation, misunderstanding or ambiguities from linking to a larger specification can be avoided. + +Grammars are made up of rules in the following form: + +``` +symbol ::= expression +``` + +We can say we have the production identified by symbol if the input is matched by the expression. +Whitespace is completely ignored in rule definitions. + +## Expressions + +The simplest expression is the literal, surrounded by quotes: + +``` +literal ::= "matchthis" +``` + +The above expression defines a symbol, "literal", that matches the exact input of "matchthis". +Character classes are delineated by brackets (`[]`), describing either a set, range or multiple range of characters: + +``` +set := [abc] +range := [A-Z] +``` + +The above symbol "set" would match one character of either "a", "b" or "c". +The symbol "range" would match any character, "A" to "Z", inclusive. +Currently, only matching for 7-bit ascii literals and character classes is defined, as that is all that is required by this specification. +Multiple character ranges and explicit characters can be specified in a single character classes, as follows: + +``` +multipleranges := [a-zA-Z=-] +``` + +The above matches the characters in the range `A` to `Z`, `a` to `z` and the individual characters `-` and `=`. + +Expressions can be made up of one or more expressions, such that one must be followed by the other. +This is known as an implicit concatenation operator. +For example, to satisfy the following rule, both `A` and `B` must be matched to satisfy the rule: + +``` +symbol ::= A B +``` + +Each expression must be matched once and only once, `A` followed by `B`. +To support the description of repetition and optional match criteria, the postfix operators `*` and `+` are defined. +`*` indicates that the preceding expression can be matched zero or more times. +`+` indicates that the preceding expression must be matched one or more times. +These appear in the following form: + +``` +zeroormore ::= expression* +oneormore ::= expression+ +``` + +Parentheses are used to group expressions into a larger expression: + +``` +group ::= (A B) +``` + +Like simpler expressions above, operators can be applied to groups, as well. +To allow for alternates, we also define the infix operator `|`. + +``` +oneof ::= A | B +``` + +The above indicates that the expression should match one of the expressions, `A` or `B`. + +## Precedence + +The operator precedence is in the following order: + +- Terminals (literals and character classes) +- Grouping `()` +- Unary operators `+*` +- Concatenation +- Alternates `|` + +The precedence can be better described using grouping to show equivalents. +Concatenation has higher precedence than alernates, such `A B | C D` is equivalent to `(A B) | (C D)`. +Unary operators have higher precedence than alternates and concatenation, such that `A+ | B+` is equivalent to `(A+) | (B+)`. + +## Examples + +The following combines the previous definitions to match a simple, relative path name, describing the individual components: + +``` +path ::= component ("/" component)* +component ::= [a-z]+ +``` + +The production "component" is one or more lowercase letters. +A "path" is then at least one component, possibly followed by zero or more slash-component pairs. +The above can be converted into the following regular expression: + +``` +[a-z]+(?:/[a-z]+)* +``` + +[ebnf]: https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form +[xmlebnf]: https://www.w3.org/TR/REC-xml/#sec-notation +[canonical-json]: http://wiki.laptop.org/go/Canonical_JSON +[github.com/docker/go]: https://github.com/docker/go/ +[Go]: https://golang.org/ +[JSON]: http://json.org/ diff --git a/delete.md b/delete.md new file mode 100644 index 0000000..79a790c --- /dev/null +++ b/delete.md @@ -0,0 +1,13 @@ +To clarify the current high-level differences with the artifact-manifest and the existing image-manifest: + +| Image-spec | Artifacts Spec | +|-|-| +| `config` REQUIRED | `config` optional as it's just another entry in the `blobs` collection with a `config mediaType` | +| `layers` REQUIRED | `blobs`, which renamed `layers` to reflect general usage are OPTIONAL | +| `layers` ORDINAL | `blobs` are defined by the specific artifact spec. Helm isn't ordinal, while other artifact types, like container images MAY make them ordinal | +| `manifest.config.mediaType` used to uniquely identify different artifact types. | `manifest.artifactType` added to lift the workaround for using `manifest.config.mediaType` on a REQUIRED, but not always used property. | +| | `subjectManifest` OPTIONAL, enabling an artifact to extend another artifact (SBOM, Signatures, Nydus, Scan Results, ) +| | `/referrers` api for discovering referenced artifacts, with the ability to filter by `artifactType` | +| | Lifecycle management defined, starting to provide standard expectations for how users can manage their content. It doesn't define GC as an internal detail| + +The artifact manifest approach to reference types is based on a new manifest, enabling registries and clients to opt-into the behavior, with clear and consistent expectations, rather than slipping new content into a registry, or client, that may, or may not know how to lifecycle manage the new content. See [Discussion of a new manifest #41](https://github.com/opencontainers/artifacts/discussions/41) diff --git a/descriptor.md b/descriptor.md new file mode 100644 index 0000000..194d223 --- /dev/null +++ b/descriptor.md @@ -0,0 +1,183 @@ +# OCI Content Descriptors + +* An OCI artifact consists of several different components, arranged in a [Merkle Directed Acyclic Graph (DAG)](https://en.wikipedia.org/wiki/Merkle_tree). +* References between components in the graph are expressed through _Content Descriptors_. +* A Content Descriptor (or simply _Descriptor_) describes the disposition of the targeted content. +* A Content Descriptor includes the type of the content, a content identifier (_digest_), and the byte-size of the raw content. +* Descriptors SHOULD be embedded in other formats to securely reference external content. +* Other formats SHOULD use descriptors to securely reference external content. + +This section defines the `application/vnd.oci.descriptor.v1+json` media type. + +## Properties + +A descriptor consists of a set of properties encapsulated in key-value fields. + +The following fields contain the primary properties that constitute a Descriptor: + +- **`mediaType`** *string* + + This REQUIRED property contains the media type of the referenced content. + Values MUST comply with [RFC 6838][rfc6838], including the [naming requirements in its section 4.2][rfc6838-s4.2]. + + Each artifact author MAY define their own unique `mediaTypes`, or utilize existing `mediaTypes` defined by other artifacts. To assure unique ownership, all `mediaTypes` MUST be registered with iana.org. + +- **`digest`** *string* + + This REQUIRED property is the _digest_ of the targeted content, conforming to the requirements outlined in [Digests](#digests). + Retrieved content SHOULD be verified against this digest when consumed via untrusted sources. + +- **`size`** *int64* + + This REQUIRED property specifies the size, in bytes, of the raw content. + This property exists so that a client will have an expected size for the content before processing. + If the length of the retrieved content does not match the specified length, the content SHOULD NOT be trusted. + +- **`urls`** *array of strings* + + This OPTIONAL property specifies a list of URIs from which this object MAY be downloaded. + Each entry MUST conform to [RFC 3986][rfc3986]. + Entries SHOULD use the `http` and `https` schemes, as defined in [RFC 7230][rfc7230-s2.7]. + +- **`annotations`** *string-string map* + + This OPTIONAL property contains arbitrary metadata for this descriptor. + This OPTIONAL property MUST use the [annotation rules](annotations.md#rules). + +### Reserved + +The following field keys are reserved and MUST NOT be used by other specifications. + +- **`data`** *string* + + This key is RESERVED for future versions of the specification. + +All other fields may be included in other OCI specifications. +Extended _Descriptor_ field additions proposed in other OCI specifications SHOULD first be considered for addition into this specification. + +## Digests + +The _digest_ property of a Descriptor acts as a content identifier, enabling [content addressability](http://en.wikipedia.org/wiki/Content-addressable_storage). +It uniquely identifies content by taking a [collision-resistant hash](https://en.wikipedia.org/wiki/Cryptographic_hash_function) of the bytes. +If the _digest_ can be communicated in a secure manner, one can verify content from an insecure source by recalculating the digest independently, ensuring the content has not been modified. + +The value of the `digest` property is a string consisting of an _algorithm_ portion and an _encoded_ portion. +The _algorithm_ specifies the cryptographic hash function and encoding used for the digest; the _encoded_ portion contains the encoded result of the hash function. + +A digest string MUST match the following [grammar](considerations.md#ebnf): + +``` +digest ::= algorithm ":" encoded +algorithm ::= algorithm-component (algorithm-separator algorithm-component)* +algorithm-component ::= [a-z0-9]+ +algorithm-separator ::= [+._-] +encoded ::= [a-zA-Z0-9=_-]+ +``` + +Note that _algorithm_ MAY impose algorithm-specific restriction on the grammar of the _encoded_ portion. +See also [Registered Algorithms](#registered-algorithms). + +Some example digest strings include the following: + +digest | algorithm | Registered | +--------------------------------------------------------------------------|---------------------|------------| +`sha256:6c3c624b58dbbcd3c0dd82b4c53f04194d1247c6eebdaab7c610cf7d66709b3b` | [SHA-256](#sha-256) | Yes | +`sha512:401b09eab3c013d4ca54922bb802bec8fd5318192b0a75f201d8b372742...` | [SHA-512](#sha-512) | Yes | +`multihash+base58:QmRZxt2b1FVZPNqd8hsiykDL3TdBDeTSPX9Kv46HmX4Gx8` | Multihash | No | +`sha256+b64u:LCa0a2j_xo_5m0U8HTBBNBNCLXBkg7-g-YpeiGJm564` | SHA-256 with urlsafe base64 | No | + +Please see [Registered Algorithms](#registered-algorithms) for a list of registered algorithms. + +Implementations SHOULD allow digests with unrecognized algorithms to pass validation if they comply with the above grammar. +While `sha256` will only use hex encoded digests, separators in _algorithm_ and alphanumerics in _encoded_ are included to allow for extensions. +As an example, we can parameterize the encoding and algorithm as `multihash+base58:QmRZxt2b1FVZPNqd8hsiykDL3TdBDeTSPX9Kv46HmX4Gx8`, which would be considered valid but unregistered by this specification. + +### Verification + +Before consuming content targeted by a descriptor from untrusted sources, the byte content SHOULD be verified against the digest string. +Before calculating the digest, the size of the content SHOULD be verified to reduce hash collision space. +Heavy processing before calculating a hash SHOULD be avoided. +Implementations MAY employ [canonicalization](considerations.md#canonicalization) of the underlying content to ensure stable content identifiers. + +### Digest calculations + +A _digest_ is calculated by the following pseudo-code, where `H` is the selected hash algorithm, identified by string ``: +``` +let ID(C) = Descriptor.digest +let C = +let D = ':' + Encode(H(C)) +let verified = ID(C) == D +``` +Above, we define the content identifier as `ID(C)`, extracted from the `Descriptor.digest` field. +Content `C` is a string of bytes. +Function `H` returns the hash of `C` in bytes and is passed to function `Encode` and prefixed with the algorithm to obtain the digest. +The result `verified` is true if `ID(C)` is equal to `D`, confirming that `C` is the content identified by `D`. +After verification, the following is true: + +``` +D == ID(C) == ':' + Encode(H(C)) +``` + +The _digest_ is confirmed as the content identifier by independently calculating the _digest_. + +### Registered algorithms + +While the _algorithm_ component of the digest string allows the use of a variety of cryptographic algorithms, compliant implementations SHOULD use [SHA-256](#sha-256). + +The following algorithm identifiers are currently defined by this specification: + +| algorithm identifier | algorithm | +|----------------------|---------------------| +| `sha256` | [SHA-256](#sha-256) | +| `sha512` | [SHA-512](#sha-512) | + +If a useful algorithm is not included in the above table, it SHOULD be submitted to this specification for registration. + +#### SHA-256 + +[SHA-256][rfc4634-s4.1] is a collision-resistant hash function, chosen for ubiquity, reasonable size and secure characteristics. +Implementations MUST implement SHA-256 digest verification for use in descriptors. + +When the _algorithm identifier_ is `sha256`, the _encoded_ portion MUST match `/[a-f0-9]{64}/`. +Note that `[A-F]` MUST NOT be used here. + +#### SHA-512 + +[SHA-512][rfc4634-s4.2] is a collision-resistant hash function which [may be more perfomant][sha256-vs-sha512] than [SHA-256](#sha-256) on some CPUs. +Implementations MAY implement SHA-512 digest verification for use in descriptors. + +When the _algorithm identifier_ is `sha512`, the _encoded_ portion MUST match `/[a-f0-9]{128}/`. +Note that `[A-F]` MUST NOT be used here. + +## Examples + +The following example describes a manifest with a content identifier of "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270" and a size of 7682 bytes: + +```json,title=Content%20Descriptor&mediatype=application/vnd.oci.descriptor.v1%2Bjson +{ + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "size": 7682, + "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270" +} +``` + +In the following example, the descriptor indicates that the referenced manifest is retrievable from a particular URL: + +```json,title=Content%20Descriptor&mediatype=application/vnd.oci.descriptor.v1%2Bjson +{ + "mediaType": "application/vnd.oci.artifact.manifest.v1+json", + "size": 7682, + "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270", + "urls": [ + "https://example.com/example-manifest" + ] +} +``` + +[rfc3986]: https://tools.ietf.org/html/rfc3986 +[rfc4634-s4.1]: https://tools.ietf.org/html/rfc4634#section-4.1 +[rfc4634-s4.2]: https://tools.ietf.org/html/rfc4634#section-4.2 +[rfc6838]: https://tools.ietf.org/html/rfc6838 +[rfc6838-s4.2]: https://tools.ietf.org/html/rfc6838#section-4.2 +[rfc7230-s2.7]: https://tools.ietf.org/html/rfc7230#section-2.7 +[sha256-vs-sha512]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/hsMw7cAwrZE diff --git a/manifest-referrers-api.md b/manifest-referrers-api.md new file mode 100644 index 0000000..51f644d --- /dev/null +++ b/manifest-referrers-api.md @@ -0,0 +1,109 @@ +# Manifest Referrers API + +[OCI artifact-manifest](./artifact-manifest.md) provides the ability to reference artifacts to existing artifacts. Reference artifacts include Notary v2 signatures, SBoMs and many other types. Artifacts that reference other artifacts SHOULD NOT be tagged, as they are considered enhancements to the artifacts they reference. To discover referenced artifacts a manifest referrers API is provided. An artifact client, such as a Notary v2 client would parse the returned manifests, determining which manifest type they will pull and process. + +The `referrers` API returns all artifacts that have a `subjectManifest` to given manifest digest. Referenced artifact requests are scoped to a repository, ensuring access rights for the repository can be used as authorization for the referenced artifacts. + +Artifact references are defined in the [oci.artifact.manifest spec][oci.artifact.manifest-spec] through the [`subjectManifest`][oci.artifact.manifest-spec-manifests] property. + +## Request All Artifact References + +The references api is defined as an extension to the [distribution-spec][oci-distribution-spec] using the `/v2/_ext/oci-artifacts/v1-rc1/` namespace. This spec defines the behavior of the `v1-rc1` version. Clients MUST account for version checking as future major versions MAY NOT be compatible. Future Minor versions MUST be additive. + +The `/referrers` API MUST provide for paging. The default page size SHOULD be set to 10. + +```rest +GET /v2/_ext/oci-artifacts/v1-rc1/{repository}/manifests/{digest}/referrers?n=10 +``` + +**expanded example:** + +```rest +GET /v2/_ext/oci-artifacts/v1-rc1/net-monitor/manifests/sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b/referrers?n=10 +``` + +The `/referrers` API MAY provide for filtering of `artifactTypes`. Artifact clients MUST account for [distribution-spec][oci-distribution-spec] implementations that MAY NOT support filtering. Artifact clients MUST revert to client side filtering to determine which `artifactTypes` they will process. + +### Request Artifacts of a specific media type + +**template:** +```rest +GET /v2/_ext/oci-artifacts/v1-rc1/{repository}/manifests/{digest}/referrers?n=10&artifactType={artifactType} +``` + +**expanded example:** + +```rest +GET /v2/_ext/oci-artifacts/v1-rc1/net-monitor/manifests/sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b/referrers?n=10&artifactType=application/vnd.oci.notary.v2 +``` + +### Artifact Referrers API results + +[distribution-spec][oci-distribution-spec] implementations MAY implement `artifactType` filtering. Some artifacts types including Notary v2 signatures, may return multiple signatures of the same `artifactType`. To avoid an artifact client from having to retrieve each manifest, just to determine if it's the specific artifact needed to continue processing, the `/referrers` API will return a collection of manifests, including the annotations within each manifest. By providing manifests, as opposed to manifest descriptors, a specific artifact client can find the relevant properties they need to determine which artifact to retrieve. For example, Notary v2 MAY use an annotation: `"org.cncf.notary.v2.signature.subject": "wabbit-networks.io"`, which the client could use to determine which signature to pull from the registry. Using annotations, clients can reduce round trips and the data returned to determine which artifacts the specific client may require reducing network traffic and API calls. + +This paged result MUST return the following elements: + +- `digest`: The digest used to retrieve the referenced manifest +- `manifest`: The [pretty](https://linuxhint.com/pretty_json_php/) listing of `oci.artifact.manifest`. By providing the manifest, consumers can choose which elements they require to determine which manifest of the paged result they must fetch. As the manifest is pretty formatted, the contents MAY not match the digest of the original manifest. For to assure the contents are accurate, the client MAY retrieve the manifest using `references.[n].digest` +- `@nextLink`: Used for paged results + +As an example, Notary v2 manifests use annotations to determine which Notary v2 signature they should retrieve: `"org.cncf.notary.v2.signature.subject": "wabbit-networks.io"` + +**example result of artifacts that reference the `net-monitor` image:** +```json +{ + "references": [ + { + "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b", + "manifest": { + "schemaVersion": 3, + "mediaType": "application/vnd.oci.artifact.manifest.v1-rc1+json", + "artifactType": "cncf.notary.v2-rc1", + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b", + "size": 16724 + }, + "annotations": { + "org.cncf.notary.v2.signature.subject": "wabbit-networks.io" + } + } + }, + { + "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b", + "manifest": { + "schemaVersion": 1, + "mediaType": "application/vnd.oci.artifact.manifest.v1-rc1+json", + "artifactType": "example.sbom.v0" + }, + "blobs": [ + { + "mediaType": "application/tar", + "digest": "sha256:9834876dcfb05cb167a5c24953eba58c4ac89b1adf57f28f2f9d09af107ee8f0", + "size": 32654 + } + ], + "subjectManifest": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b", + "size": 16724 + }, + "annotations": { + "example.sbom.author": "wabbit-networks.io" + } + } + ], + "@nextLink": "{opaqueUrl}" +} +``` + +[oci.artifact.manifest-spec]: ./artifact-manifest-spec.md +[oci.artifact.manifest-spec-manifests]: ./artifact-manifest-spec.md#oci-artifact-manifest-properties +[oci-distribution-spec]: https://github.com/opencontainers/distribution-spec diff --git a/media/artifacts-spec-objects.png b/media/artifacts-spec-objects.png index b422d40..8cacfb7 100644 Binary files a/media/artifacts-spec-objects.png and b/media/artifacts-spec-objects.png differ diff --git a/media/image-spec-objects.png b/media/image-spec-objects.png index f9b1648..a233329 100644 Binary files a/media/image-spec-objects.png and b/media/image-spec-objects.png differ diff --git a/media/net-monitor-copy.svg b/media/net-monitor-copy.svg new file mode 100644 index 0000000..7ab0cc6 --- /dev/null +++ b/media/net-monitor-copy.svg @@ -0,0 +1 @@ +ACME RocketsArtifact Copynet-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestnet-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifest \ No newline at end of file diff --git a/media/net-monitor-graph.svg b/media/net-monitor-graph.svg new file mode 100644 index 0000000..d3de31f --- /dev/null +++ b/media/net-monitor-graph.svg @@ -0,0 +1 @@ +net-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestwabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2SBoMDocumentsbom[blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: example.sbom.v0wabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2 \ No newline at end of file diff --git a/media/net-monitor-image.svg b/media/net-monitor-image.svg new file mode 100644 index 0000000..9c36d03 --- /dev/null +++ b/media/net-monitor-image.svg @@ -0,0 +1 @@ +net-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifest \ No newline at end of file diff --git a/media/net-monitor-sbom-signature.svg b/media/net-monitor-sbom-signature.svg new file mode 100644 index 0000000..8a8b7f9 --- /dev/null +++ b/media/net-monitor-sbom-signature.svg @@ -0,0 +1 @@ +wabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2SBoMDocumentsbom[blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: example.sbom.v0 \ No newline at end of file diff --git a/media/net-monitor-sbom.svg b/media/net-monitor-sbom.svg new file mode 100644 index 0000000..2bfd764 --- /dev/null +++ b/media/net-monitor-sbom.svg @@ -0,0 +1 @@ +net-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestSBoMDocumentsbom[blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: example.sbom.v0 \ No newline at end of file diff --git a/media/net-monitor-with-sigs-copy.svg b/media/net-monitor-with-sigs-copy.svg new file mode 100644 index 0000000..9ee5f27 --- /dev/null +++ b/media/net-monitor-with-sigs-copy.svg @@ -0,0 +1 @@ +Artifact CopyACME Rocketsnet-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestwabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2SBoMDocumentsbom[blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: example.sbom.v0wabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2net-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestwabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2SBoMDocumentsbom[blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: example.sbom.v0wabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2 \ No newline at end of file diff --git a/media/notaryv2-signature.svg b/media/notaryv2-signature.svg new file mode 100644 index 0000000..b9e8913 --- /dev/null +++ b/media/notaryv2-signature.svg @@ -0,0 +1 @@ +net-monitor:v1layer1 (blob)layer2 (blob)config (blob)mediaType: oci.image.manifestwabbit-networks signaturesignature [blobs]reference (subjectManifest)mediaType: oci.artifact.manifest.v1artifactType: cncf.notary.v2 \ No newline at end of file diff --git a/media/repo-listing-attributed-expanded.svg b/media/repo-listing-attributed-expanded.svg new file mode 100644 index 0000000..19f8603 --- /dev/null +++ b/media/repo-listing-attributed-expanded.svg @@ -0,0 +1 @@ +v1………………chart-v1Tag countManifest countLast updated dateRepositorynet-monitor5/14/2021, 2:43 PM PDT26Tags8ac803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c222SBoM7a781a3930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c1aea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae1473c803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c333 \ No newline at end of file diff --git a/media/repo-listing-attributed.svg b/media/repo-listing-attributed.svg new file mode 100644 index 0000000..2fda9fd --- /dev/null +++ b/media/repo-listing-attributed.svg @@ -0,0 +1 @@ +v1……chart-v1Tag countManifest countLast updated dateRepositorynet-monitor5/14/2021, 2:43 PM PDT26TagsSBoM \ No newline at end of file diff --git a/media/repo-listing-flat.svg b/media/repo-listing-flat.svg new file mode 100644 index 0000000..4bec0b2 --- /dev/null +++ b/media/repo-listing-flat.svg @@ -0,0 +1 @@ +v1SBoM………………8ac803930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c2227a781a3930ea3ba1e54bc25c2bdc53edd0284c62ed651fe7b00369da519a3c1aea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae14chart-v1ea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae14Tag countManifest countLast updated dateRepositorynet-monitor5/14/2021, 2:43 PM PDT26Tags \ No newline at end of file