Skip to content

Commit

Permalink
Support Cluster ImagePolicy CRD
Browse files Browse the repository at this point in the history
Signed-off-by: Qi Wang <qiwan@redhat.com>
  • Loading branch information
QiWang19 committed Feb 3, 2024
1 parent 2464fef commit d31d7ce
Show file tree
Hide file tree
Showing 18 changed files with 1,889 additions and 125 deletions.
1 change: 1 addition & 0 deletions cmd/machine-config-controller/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ func createControllers(ctx *ctrlcommon.ControllerContext) []ctrlcommon.Controlle
ctx.ConfigInformerFactory.Config().V1().Images(),
ctx.ConfigInformerFactory.Config().V1().ImageDigestMirrorSets(),
ctx.ConfigInformerFactory.Config().V1().ImageTagMirrorSets(),
ctx.ConfigInformerFactory.Config().V1alpha1().ClusterImagePolicies(),
ctx.OperatorInformerFactory.Operator().V1alpha1().ImageContentSourcePolicies(),
ctx.ConfigInformerFactory.Config().V1().ClusterVersions(),
ctx.ClientBuilder.KubeClientOrDie("container-runtime-config-controller"),
Expand Down
48 changes: 48 additions & 0 deletions docs/ClusterImagePolicyDesign.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
## Summary
ClusterImagePolicy and ImagePolicy are CRDs that managed by ContainerRuntimeConfig controller. These CRDs allow setting up configurations for CRI-O to verify the container images signed using [Sigstore](https://www.sigstore.dev/) tools.

## Goals
Generating corresponding CRI-O configuration files for image signature verification. Rollout ClusterImagePolicy to `/etc/containers/policy.json` for cluster wide configuration. Roll out ImagePolicy to `/etc/crio/policies/<NAMESPACE>.json` for pod namespace-separated signature policies configuration.

## Non-Goals
Rolling out configuration for OCP payload repositories. The (super scope of) OCP payload repositories will not be written to the configuration files.

## CRD
[ClusterImagePolicy CRD](https://github.com/openshift/api/blob/master/config/v1alpha1/0000_10_config-operator_01_clusterimagepolicy-TechPreviewNoUpgrade.crd.yaml)

[ImagePolicy CRD](https://github.com/openshift/api/blob/master/config/v1alpha1/0000_10_config-operator_01_imagepolicy-TechPreviewNoUpgrade.crd.yaml)

## Example

## Validation and Troubleshooting

## Implementation Details
The ContainerRuntimeConfigController would perform the following steps:

1. Validate the ClusterImagePolicy and ImagePolicy objects on the cluster. Follow the table below to ignore the conflicting scopes.

| |process the policies from the CRs | | | |
|----------------------------------------------------------------------------------------------------------------- |------------------------------------------------ |----------------------------------------------------------------------------------- |--- |--- |
| same scope in different CRs | ImagePolicy | ClusterImagePolicy | | |
| ClusterImagePolicy ImagePolicy (scope in the ClusterImagePolicy is equal to or broader than in the ImagePolicy) | Do not deploy non-global policy for this scope | Write the cluster policy to `/etc/containers/policy.json` and `<NAMESPACE>.json` | | |
| ClusterImagePolicy ClusterImagePolicy | N/A | Append the policy to existing `etc/containers/policy.json` | | |
| ImagePolicy ImagePolicy | append the policy to <NAMESPACE>.json | N/A | | |

2. Render the current MachineConfigs (storage.files.contents[policy.json]) into the originalPolicyIgn

3. Serialize the cluster level policies to `policy.json`.

4. Copy the cluster policy.json to `<NAMESPACE>.json`, serialize the namespace level policies to `<NAMESPACE>.json`.

5. Add registries configuration to `/etc/containers/registries.d/sigstore-registries.yaml`. This configuration is used to specify the sigstore is being used as the image signature verification backend.

6. Update the ignition file `/etc/containers/policy.json` within the `99-<pool>-generated-registries` MachineConfig.

7. Create or Update the ignition file `/etc/crio/policies/<NAMESPACE>.json` within the `99-<pool>-generated-imagepolicies` MachineConfig.

After deletion all of the ClusterImagePolicy or the ImagePolicy instance the config will be reverted to the original policy.json.

## See Also
see **[containers-policy.json(5)](https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md)**, **[containers-registries.d(5)](https://github.com/containers/image/blob/main/docs/containers-registries.d.5.md)** for more information.


Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,13 @@ rules:
- get
- list
- watch
- apiGroups:
- config.openshift.io
resources:
- imagepolicies
- clusterimagepolicies
verbs:
- get
- list
- watch
- update
2 changes: 1 addition & 1 deletion manifests/machineconfigcontroller/clusterrole.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ rules:
resources: ["configmaps", "secrets"]
verbs: ["*"]
- apiGroups: ["config.openshift.io"]
resources: ["images", "clusterversions", "featuregates", "nodes", "nodes/status"]
resources: ["images", "clusterversions", "featuregates", "nodes", "nodes/status", "imagepolicies", "imagepolicies/status", "clusterimagepolicies", "clusterimagepolicies/status"]
verbs: ["*"]
- apiGroups: ["config.openshift.io"]
resources: ["schedulers", "apiservers", "infrastructures", "imagedigestmirrorsets", "imagetagmirrorsets"]
Expand Down
10 changes: 10 additions & 0 deletions pkg/apihelpers/apihelpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,16 @@ func NewKubeletConfigCondition(condType mcfgv1.KubeletConfigStatusConditionType,
}
}

func NewCondition(condType string, status metav1.ConditionStatus, reason, message string) *metav1.Condition {
return &metav1.Condition{
Type: condType,
Status: status,
LastTransitionTime: metav1.Now(),
Reason: reason,
Message: message,
}
}

// NewContainerRuntimeConfigCondition returns an instance of a ContainerRuntimeConfigCondition
func NewContainerRuntimeConfigCondition(condType mcfgv1.ContainerRuntimeConfigStatusConditionType, status corev1.ConditionStatus, message string) *mcfgv1.ContainerRuntimeConfigCondition {
return &mcfgv1.ContainerRuntimeConfigCondition{
Expand Down
35 changes: 21 additions & 14 deletions pkg/controller/bootstrap/bootstrap.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"k8s.io/klog/v2"

apicfgv1 "github.com/openshift/api/config/v1"
apicfgv1alpha1 "github.com/openshift/api/config/v1alpha1"
mcfgv1 "github.com/openshift/api/machineconfiguration/v1"
apioperatorsv1alpha1 "github.com/openshift/api/operator/v1alpha1"
"github.com/openshift/library-go/pkg/operator/configobserver/featuregates"
Expand Down Expand Up @@ -72,20 +73,24 @@ func (b *Bootstrap) Run(destDir string) error {
mcfgv1.Install(scheme)
apioperatorsv1alpha1.Install(scheme)
apicfgv1.Install(scheme)
apicfgv1alpha1.Install(scheme)
codecFactory := serializer.NewCodecFactory(scheme)
decoder := codecFactory.UniversalDecoder(mcfgv1.GroupVersion, apioperatorsv1alpha1.GroupVersion, apicfgv1.GroupVersion)

var cconfig *mcfgv1.ControllerConfig
var featureGate *apicfgv1.FeatureGate
var nodeConfig *apicfgv1.Node
var kconfigs []*mcfgv1.KubeletConfig
var pools []*mcfgv1.MachineConfigPool
var configs []*mcfgv1.MachineConfig
var crconfigs []*mcfgv1.ContainerRuntimeConfig
var icspRules []*apioperatorsv1alpha1.ImageContentSourcePolicy
var idmsRules []*apicfgv1.ImageDigestMirrorSet
var itmsRules []*apicfgv1.ImageTagMirrorSet
var imgCfg *apicfgv1.Image
decoder := codecFactory.UniversalDecoder(mcfgv1.GroupVersion, apioperatorsv1alpha1.GroupVersion, apicfgv1.GroupVersion, apicfgv1alpha1.GroupVersion)

var (
cconfig *mcfgv1.ControllerConfig
featureGate *apicfgv1.FeatureGate
nodeConfig *apicfgv1.Node
kconfigs []*mcfgv1.KubeletConfig
pools []*mcfgv1.MachineConfigPool
configs []*mcfgv1.MachineConfig
crconfigs []*mcfgv1.ContainerRuntimeConfig
icspRules []*apioperatorsv1alpha1.ImageContentSourcePolicy
idmsRules []*apicfgv1.ImageDigestMirrorSet
itmsRules []*apicfgv1.ImageTagMirrorSet
clusterImagePolicies []*apicfgv1alpha1.ClusterImagePolicy
imgCfg *apicfgv1.Image
)
for _, info := range infos {
if info.IsDir() {
continue
Expand Down Expand Up @@ -132,6 +137,8 @@ func (b *Bootstrap) Run(destDir string) error {
itmsRules = append(itmsRules, obj)
case *apicfgv1.Image:
imgCfg = obj
case *apicfgv1alpha1.ClusterImagePolicy:
clusterImagePolicies = append(clusterImagePolicies, obj)
case *apicfgv1.FeatureGate:
if obj.GetName() == ctrlcommon.ClusterFeatureInstanceName {
featureGate = obj
Expand Down Expand Up @@ -168,7 +175,7 @@ func (b *Bootstrap) Run(destDir string) error {

configs = append(configs, iconfigs...)

rconfigs, err := containerruntimeconfig.RunImageBootstrap(b.templatesDir, cconfig, pools, icspRules, idmsRules, itmsRules, imgCfg, fgAccess)
rconfigs, err := containerruntimeconfig.RunImageBootstrap(b.templatesDir, cconfig, pools, icspRules, idmsRules, itmsRules, imgCfg, clusterImagePolicies, fgAccess)
if err != nil {
return err
}
Expand Down
2 changes: 2 additions & 0 deletions pkg/controller/bootstrap/testdata/bootstrap/featuregate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ status:
- version: 0.0.1-snapshot
enabled:
- name: OpenShiftPodSecurityAdmission
disabled:
- name: SigstoreImageVerification
Loading

0 comments on commit d31d7ce

Please sign in to comment.