Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Platform api changes to support phase 1 of Dockerfiles implementation #308

Merged
merged 14 commits into from
Aug 23, 2022
Merged

Platform api changes to support phase 1 of Dockerfiles implementation #308

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
42a62e9
Platform api changes to support phase 1 of Dockerfiles implementation
natalieparellano Apr 15, 2022
e669a53
Add extensions to build metadata
natalieparellano May 2, 2022
9867543
Updates from implementation
natalieparellano Jun 15, 2022
da18b78
Add extension to build plan
natalieparellano Jun 15, 2022
b8e6b13
Updates per recent Working Group discussions
natalieparellano Jun 24, 2022
1d79fcb
Update with information about experimental features
natalieparellano Jun 24, 2022
251c7a0
Fix generate exit code
natalieparellano Jun 29, 2022
b900760
Update env var name
natalieparellano Jul 7, 2022
642eaf4
Update platform.md
natalieparellano Jul 12, 2022
e9b350a
Update platform.md
natalieparellano Jul 12, 2022
be44080
Update platform.md
natalieparellano Jul 12, 2022
e8dda22
Clarify the meaning of `extension` in plan.toml
natalieparellano Jul 12, 2022
a5acc10
Update platform.md
natalieparellano Aug 16, 2022
2d73272
Merge branch 'main' into extensions-platform-api
natalieparellano Aug 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 121 additions & 29 deletions platform.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,25 +42,30 @@ Examples of a platform might include:
- [Inputs](#inputs-2)
- [Outputs](#outputs-2)
- [Layer Restoration](#layer-restoration)
- [`builder`](#builder)
- [`generator`](#generator)
- [Inputs](#inputs-3)
- [Outputs](#outputs-3)
- [`exporter`](#exporter)
- [`builder`](#builder)
- [Inputs](#inputs-4)
- [Outputs](#outputs-4)
- [`creator`](#creator)
- [`exporter`](#exporter)
- [Inputs](#inputs-5)
- [Outputs](#outputs-5)
- [`rebaser`](#rebaser)
- [`creator`](#creator)
- [Inputs](#inputs-6)
- [Outputs](#outputs-6)
- [`launcher`](#launcher)
- [`rebaser`](#rebaser)
- [Inputs](#inputs-7)
- [Outputs](#outputs-7)
- [`launcher`](#launcher)
- [Inputs](#inputs-8)
- [Outputs](#outputs-8)
- [Run Image Resolution](#run-image-resolution)
- [Registry Authentication](#registry-authentication)
- [Buildpacks](#buildpacks)
- [Buildpacks Directory Layout](#buildpacks-directory-layout)
- [Image Extensions](#image-extensions)
- [Image Extensions Directory Layout](#image-extensions-directory-layout)
- [Security Considerations](#security-considerations)
- [Additional Guidance](#additional-guidance)
- [Environment](#environment)
Expand All @@ -73,6 +78,7 @@ Examples of a platform might include:
- [Data Format](#data-format)
- [Files](#files)
- [`analyzed.toml` (TOML)](#analyzedtoml-toml)
- [`extender.toml` (TOML)](#extendertoml-toml)
- [`group.toml` (TOML)](#grouptoml-toml)
- [`metadata.toml` (TOML)](#metadatatoml-toml)
- [`order.toml` (TOML)](#ordertoml-toml)
Expand Down Expand Up @@ -122,6 +128,8 @@ A **launcher layer** refers to a layer in the app OCI image containing the **lau

The **launcher** refers to a lifecycle executable packaged in the **app image** for the purpose of executing processes at runtime.

An **image extension** refers to software compliant with the [Image Extension Interface Specification](image-extension.md). Image extensions participate in detection and execute before the buildpack build process.

#### Additional Terminology
An **image reference** refers to either a **tag reference** or **digest reference**.

Expand Down Expand Up @@ -334,13 +342,13 @@ Usage:
| `/dev/stderr` | Logs (warnings, errors)
| `<analyzed>` | Analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml)

| Exit Code | Result|
|-----------|-------|
| `0` | Success
| `11` | Platform API incompatibility error
| `12` | Buildpack API incompatibility error
| `1-10`, `13-99` | Generic lifecycle errors
| `30-39` | Analysis-specific lifecycle errors
| Exit Code | Result|
|-----------------|-------|
| `0` | Success
| `11` | Platform API incompatibility error
| `12` | Buildpack API incompatibility error
| `1-10`, `13-19` | Generic lifecycle errors
| `30-39` | Analysis-specific lifecycle errors

- The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to `<analyzed>`, where:
- `image` MUST describe the `<previous-image>`, if accessible
Expand All @@ -354,6 +362,7 @@ Usage:
/cnb/lifecycle/detector \
[-app <app>] \
[-buildpacks <buildpacks>] \
[-exts <exts>] \
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
[-group <group>] \
[-layers <layers>] \
[-log-level <log-level>] \
Expand All @@ -363,16 +372,17 @@ Usage:
```

##### Inputs
| Input | Environment Variable | Default Value | Description
|---------------|-------------------------|--------------------------------------------------------|-------
| `<app>` | `CNB_APP_DIR` | `/workspace` | Path to application directory
| `<buildpacks>` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout))
| `<group>` | `CNB_GROUP_PATH` | `<layers>/group.toml` | Path to output group definition
| `<layers>` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory
| `<log-level>` | `CNB_LOG_LEVEL` | `info` | Log Level
| `<order>` | `CNB_ORDER_PATH` | `<layers>/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml))
| `<plan>` | `CNB_PLAN_PATH` | `<layers>/plan.toml` | Path to output resolved build plan
| `<platform>` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory
| Input | Environment Variable | Default Value | Description |
|----------------|----------------------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|
| `<app>` | `CNB_APP_DIR` | `/workspace` | Path to application directory |
| `<buildpacks>` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) |
| `<ext>` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) |
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
| `<group>` | `CNB_GROUP_PATH` | `<layers>/group.toml` | Path to output group definition |
| `<layers>` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory |
| `<log-level>` | `CNB_LOG_LEVEL` | `info` | Log Level |
| `<order>` | `CNB_ORDER_PATH` | `<layers>/order.toml` if present, or `/cnb/order.toml` | Path resolution for order definition (see [`order.toml`](#ordertoml-toml)) |
| `<plan>` | `CNB_PLAN_PATH` | `<layers>/plan.toml` | Path to output resolved build plan |
| `<platform>` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory |

##### Outputs
| Output | Description
Expand Down Expand Up @@ -436,13 +446,13 @@ Usage:
| `<layers>/<buidpack-id>/<layer>.sbom.<ext>` | Files containing the Software Bill of Materials for each analyzed layer (see [Buildpack Interface Specification](buildpack.md))
| `<layers>/<buidpack-id>/<layer>/*`. | Restored layer contents

| Exit Code | Result|
|-----------|-------|
| `0` | Success
| `11` | Platform API incompatibility error
| `12` | Buildpack API incompatibility error
| `1-10`, `13-99` | Generic lifecycle errors
| `40-49` | Restoration-specific lifecycle errors
| Exit Code | Result|
|-----------------|-------|
| `0` | Success
| `11` | Platform API incompatibility error
| `12` | Buildpack API incompatibility error
| `1-10`, `13-19` | Generic lifecycle errors
| `40-49` | Restoration-specific lifecycle errors

- For each buildpack in `<group>`, if persistent metadata for that buildpack exists in the analysis metadata, lifecycle MUST write a toml representation of the persistent metadata to `<layers>/<buildpack-id>/store.toml`
- **If** `<skip-layers>` is `true` the lifecycle MUST NOT perform layer restoration.
Expand All @@ -451,6 +461,60 @@ Usage:
##### Layer Restoration
lifeycle MUST use the provided `cache-dir` or `cache-image` to retrieve cache contents. The [rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#layer-types) for restoration MUST be followed when determining how and when to store cache layers.

#### `generator`
The platform MAY execute `generator` in the **build environment** for Linux builds

Usage:
```
/cnb/lifecycle/generator \
[-analyzed <analyzed>] \
[-app <app>] \
[-extender <extender> ] \
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
[-exts <exts>] \
[-group <group>] \
[-log-level <log-level>] \
[-output <output>] \
[-plan <plan>] \
[-platform <platform>]
```

##### Inputs
| Input | Env | Default Value | Description |
|---------------|----------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------|
| `<analyzed>` | `CNB_ANALYZED_PATH` | `<layers>/analyzed.toml` | Path to analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) |
| `<app>` | `CNB_APP_DIR` | `/workspace` | Path to application directory |
| `<exts>` | `CNB_IMAGE_EXTS_DIR` | `/cnb/exts` | Path to image extensions directory (see [Image Extensions Directory Layout](#image-extensions-directory-layout) |
| `<group>` | `CNB_GROUP_PATH` | `<layers>/group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) |
| `<log-level>` | `CNB_LOG_LEVEL` | `info` | Log Level |
| `<output>` | `CNB_EXT_OUTPUT_DIR` | `<layers>/exts` | Path to output directory |
| `<plan>` | `CNB_PLAN_PATH` | `<layers>/plan.toml` | Path to resolved build plan (see [`plan.toml`](#plantoml-toml)) |
| `<platform>` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory |

##### Outputs
| Output | Description |
|----------------------------------------------|-------------------------------------------------------------------------------|
| [exit status] | (see Exit Code table below for values) |
| `/dev/stdout` | Logs (info) |
| `/dev/stderr` | Logs (warnings, errors) |
| `<output>/<image extension ID>/<Dockerfile>` | Generated Dockerfiles (see [Image Extension Specfication](image-extension.md) |
| `<layers>/config/metadata.toml` | Build metadata (see [`metadata.toml`](#metadatatoml-toml)) |

| Exit Code | Result |
|-----------------|--------------------------------------|
| `0` | Success |
| `11` | Platform API incompatibility error |
| `12` | Buildpack API incompatibility error |
| `1-10`, `13-19` | Generic lifecycle errors |
| `91` | Image extension build error |
| `90`, `92-99` | Generation-specific lifecycle errors |
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved

- The lifecycle SHALL execute all image extensions in the order defined in `<group>` according to the process outlined in the [Buildpack Interface Specification](buildpack.md).
- The lifecycle SHALL replace the `run-image` reference in `<analyzed>` with the selected run image reference. The selected run image reference SHALL be the base image referenced in the Dockerfile output by the last image extension in the group.
- The lifecycle SHALL add all invoked image extensions to`<layers>/config/metadata.toml`.
- The lifecycle SHALL record all Dockerfiles in `<extender>`.
- The lifecycle SHALL copy all output Dockerfiles to `<output>`.
- The lifecycle SHALL filter the build plan with dependencies provided by image extensions.

#### `builder`
The platform MUST execute `builder` in the **build environment**

Expand Down Expand Up @@ -816,6 +880,15 @@ The buildpacks directory MUST contain unarchived buildpacks such that:
- Each top-level directory is a buildpack ID.
- Each second-level directory is a buildpack version.

## Image Extensions

### Image Extensions Directory Layout

The image extensions directory MUST contain unarchived image extensions such that:

- Each top-level directory is an image extension ID.
- Each second-level directory is an image extension version.

## Security Considerations

The platform SHOULD run each phase of the lifecycle in an isolated container to prevent untrusted app and buildpack code from accessing storage credentials needed during the export and analysis phases.
Expand Down Expand Up @@ -907,6 +980,14 @@ Where:
- `run-image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon
- `previous-image.metadata` MUST be the TOML representation of the layer [metadata label](#iobuildpackslifecyclemetadata-json)

#### `extender.toml` (TOML)

```toml
[[dockerfiles]]
extension_id = "<extension id>"
path = "<path to file in output directory>"
```

#### `group.toml` (TOML)

```toml
Expand All @@ -915,6 +996,7 @@ id = "<buildpack ID>"
version = "<buildpack version>"
api = "<buildpack API version>"
homepage = "<buildpack homepage>"
extension = false
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
```

Where:
Expand All @@ -931,6 +1013,11 @@ version = "<buildpack version>"
api = "<buildpack API version>"
optional = false

[[image-extensions]]
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
id = "<image extension ID>"
version = "<image extension version>"
api = "<image extension API version>"

[[processes]]
type = "<process type>"
command = "<command>"
Expand All @@ -955,6 +1042,11 @@ Where:
id = "<buildpack ID>"
version = "<buildpack version>"
optional = false

[[order-ext]]
[[order-ext.group]]
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
id = "<image extension ID>"
version = "<image extension version>"
```

Where:
Expand Down