Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the design document #1

Merged
merged 1 commit into from
Feb 10, 2022
Merged

Add the design document #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
Container Images Rust Crate

## Documentation
[CCv1 Image Security Design document](docs/ccv1_image_security_design.md)
[Design document](docs/design.md)
[CCv1 Image Security Design document](docs/ccv1_image_security_design.md)
102 changes: 102 additions & 0 deletions docs/design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Design and architecture

## Authors
* [Arron Wang](https://github.com/arronwy)
* [Jiang Liu](https://github.com/jiangliu)

## Design
The `image-rs` crate is a rustified and tailored version of [containers/image](https://github.com/containers/image),
to provide a small, simple, secure, lightweight and high performance OCI container image management library.
The crate development is driven by the [Confidential Containers](https://github.com/confidential-containers)
project scope but it might be used in a more generic fashion.

Currently the `image-rs` crate focuses on usage scenarios to run Confidential Containers on Kubernetes clusters with
[Kata Containers](https://katacontainers.io/). It may also be enhanced to support other Confidential container runtimes
on demand. So it implements the
[CRI Image Service](https://github.com/kubernetes/cri-api/blob/master/pkg/apis/runtime/v1alpha2/api.proto#L119)
interface to be easily integrated with the K8s ecosystem. Among the container image `Build`, `Ship` and `Run` stages,
it focuses on the `Run` stage instead of `Build` and `Ship`. `Run` stage will mostly cover `Pull` and `Unpack`
operations, which will provide interfaces to prepare container images for running in a confidential environment.
Users may use any OCI image/distribution spec compatible tools to build and distribute
encrypted/plaintext container images with or without signatures.

The `image-rs` crate supports both unencrypted and encrypted container images since
[Encrypted Container Image](https://github.com/opencontainers/artifacts/pull/15)
is mandatory for many confidential container usage scenarios. It also supports image signature verification to ensure
integrity. Depending on the threat modeling of confidential containers, it should be configurable for which types of
images are supported. The policy may like below, currently we support image level signature and layer level encryption,
whether need per-layer signature support is still TBD:

1. encrypted with signature, mandatory
2. encrypted without signature, mandatory
3. unencrypted with signature, configurable and should be enabled
4. unencrypted without signature, configurable and may be disabled
Comment on lines +30 to +33
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be quicker to understand if it was formatted as a table.

Image type Image Signature? Layer encryption? ...

Also, is this scenario supported: a signed image that contains encrypted layers and unencrypted layers? (I think this should be stated clearly, maybe using the table above).

tbh, I'm wondering if we need an additional term since "encrypted image" doesn't specify if the image is signed. How about:

  • secured image: a signed image with all its levels encrypted.

Other questions:

  • What forms of signature are supported?
  • What forms of encryption are supported?
  • Can an image contain multiple encrypted layers where each layer uses a different encryption scheme?
  • What happens if an image layer fails to decrypt? Presumably, it is "invalid", but this should be stated.
  • What happens if a signing key is revoked or expired? Again, I think this should be stated explicitly.
  • Shouldn't "the whole system" require secure NTP (NTS) to stop the possibility of the host changing its clock which could affect the ability (or the validity) of signature verification? Plus Secure DNS (DoH or DoT)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For image encryption opencontainers/artifacts#15 this PR describe the details on encrypted mediatype definitions for oci image spec.
For image signning, we still don't have detail definition yet, it is next step work for CC and upstream also have a POC: kata-containers/kata-containers#3023 and issue in attestation agent to support image signning verification: https://github.com/confidential-containers/attestation-agent/issues/24


### Goals
* Support pulling OCI/docker v2 images from registries with or without authentication
* Support downloading OCI images from local [OCI layout compatiable](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) storage
* Support OCI image decryption
* Support OCI image signature verification
* Support configurable security policies for allowed OCI images
* Support seamlessly integration with Kata Agent and Attestation Agent
* Provide a command-line utility for debug
* Explore image decryption and decompression acceleration technologies
* Explore image on demand loading technologies
* Explore technologies to share image data among containers/pods with confidentiality and integrity.


### Non-Goals
* Image push, content discovery/management operations for remote registries
* Image encryption and signing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that everything involving provisioning an image is a non-goal that would bloat image-rs but let's make sure that we provide good documentation about how a user can setup images using other tools. image-rs isn't very useful if people can't easily upload encrypted/signed images.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So maybe a "Howto" doc helps here:)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooi, is it not in scope as there are already existing tools to encrypt+sign or is that we don't want to bloat the binary? If it's the latter, we can simply build two binaries from the same source (one to encrypt/sign the other to decrypt/check) to avoid any packaging bloat.

If this project supported encrypting/signing, that would make writing unit tests easier. But at an integration level, the CI should encrypt with other tooling too! ;)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This project will leverage ocicrypt-rs to do decryption and image signing verification, and we may focuse integration test for compatiable with existing container build tools like skopeo

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We hope to focus on the most valuable features at the first stage:)
Let's get something usable then we could extend it on demand:)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @arronwy and @jiangliu here, image distribution and building should be kept out of scope for now and that should be very clear.

To make it even clearer, we could indeed describe how to do those steps with other tools and libraries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we added the document for generate test data in another PR which will describe how to do those steps, and will link them when related PR are merged.

* Image export to local storage
* Image import from other service daemon or other local storage format


### Existing Solutions
* [containers-image-proxy-rs](https://github.com/containers/containers-image-proxy-rs): a rust bindings for skopeo,
which doesn't meet our requirements of small TCB and lightweight.
* [oci-distribution](https://github.com/krustlet/oci-distribution): a crate from
[krustlet](https://github.com/krustlet/krustlet) to run wasm application with K8S. Since wasm is platform agnostic
and small code size, `oci-distribution` does not support manifest lists for different platforms and internal data
struct is not friend to generic container workload.
* [oci-registry-client](https://github.com/ecarrara/oci-registry-client): a very simple client for OCI image registries
with limited features and latest commit is one and half years ago.

These projects focus on image distribution and do not support image unpack and storage management features.

## Architecture
The following diagram demonstrates the overall architecture and how to
integrate with Kata Agent:
![Architecture](images/architecture.png)

Diagram source file: [architecture.pptx](images/architecture.pptx)

The `image-rs` crate provides an API that covers the container image pulling and management parts of the OCI specification.

For image pulling part, it will follow OCI distribution and image spec to
interact with remote registries to download container image metadata
and blob data, these info will be consumed by the image management module.
After container image is downloaded, `image-rs` will talk with remote
attestation service to verify the image signature and decrypt the image
if needed. After image is unpacked, for support image caching inside the guest,
the image format module can handle the format conversion between oci v1 and docker v2.

The image pulling module cooperates with the
[Attestation-agent](https://github.com/confidential-containers/attestation-agent)
to fetch secrets for decryption/verification from the remote attestation and key management server.

Snapshot module will prepare the rootfs together with the container
configuration and metadata, the bundles can be generated for the
runtime service to consume. The snapshot module can also support different
snapshotters which can support image on demand pulling.

As container image layers decryption and decompression can be time consuming,
image-rs will support different acceleration libraries to accelerate these process
like Intel storage acceleration library:
[isa-l](https://github.com/intel/isa-l)
[isa-l_crypto](https://github.com/intel/isa-l_crypto).


## Reference
* [Nydus Image Service](https://github.com/dragonflyoss/image-service) for data on demand loading, data deduplication and encryption.
* [stargz](https://github.com/containerd/stargz-snapshotter) for data on demand loading.
Binary file added docs/images/architecture.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/architecture.pptx
View file
Open in desktop
Binary file not shown.