Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PreserveUnknownFields: false to CRDs #1356

Merged

Conversation

julianKatz
Copy link
Contributor

@julianKatz julianKatz commented Jun 10, 2021

As described in open-policy-agent/frameworks#124,
spec.PreserveUnknownFields changed default values from true to
false (respectively) in v1beta1 and v1 CRD versions. This yields a
cosmetic issue: a CRD that was previously written as a v1beta1 CRD
will retain spec.PreserveUnknownFields: true, even when re-applied
as a v1 CRD.

This does not cause any failures, and should not block the release. It
does cause a warning, and should be fixed.

Signed-off-by: juliankatz juliankatz@google.com

@codecov-commenter
Copy link

codecov-commenter commented Jun 10, 2021

Codecov Report

Merging #1356 (85d8077) into master (02f2af9) will increase coverage by 6.13%.
The diff coverage is 43.11%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1356      +/-   ##
==========================================
+ Coverage   43.39%   49.53%   +6.13%     
==========================================
  Files          47       68      +21     
  Lines        3173     4926    +1753     
==========================================
+ Hits         1377     2440    +1063     
- Misses       1599     2140     +541     
- Partials      197      346     +149     
Flag Coverage Δ
unittests 49.53% <43.11%> (+6.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
apis/status/v1beta1/constraintpodstatus_types.go 81.81% <ø> (ø)
...tatus/v1beta1/constrainttemplatepodstatus_types.go 77.77% <ø> (ø)
apis/status/v1beta1/zz_generated.deepcopy.go 0.00% <0.00%> (ø)
pkg/audit/manager.go 0.00% <0.00%> (ø)
pkg/audit/stats_reporter.go 78.37% <ø> (ø)
pkg/controller/constraint/constraint_controller.go 4.22% <0.00%> (-0.05%) ⬇️
pkg/controller/sync/stats_reporter.go 64.00% <ø> (ø)
pkg/controller/sync/sync_controller.go 0.00% <0.00%> (ø)
pkg/metrics/exporter.go 0.00% <0.00%> (ø)
pkg/metrics/prometheus_exporter.go 64.70% <0.00%> (ø)
... and 87 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e82e5b...85d8077. Read the comment docs.

@julianKatz
Copy link
Contributor Author

@sozercan Should I add this patch to all the helmify kustomization files? Or should those be picking up patches from the config/crd/?

@maxsmythe
Copy link
Contributor

They're getting picked up. We can tell b/c they're showing up in the manifest_staging directory.

Copy link
Member

@ritazh ritazh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing the mutation crd changes.

First commit, adds PreserveUnknownFields: false to the non-mutation
CRDs.  The kustomization pipeline is in a bit of a weird state given it
being in alpha, so I'm not sure what else to add for that yet.

Signed-off-by: juliankatz <juliankatz@google.com>
Signed-off-by: juliankatz <juliankatz@google.com>
The patching was in the wrong place.  Per Rita's comment, I moved it
from:

config/overlays/mutation/kustomization.yaml

to

config/overlays/mutation_webhook/kustomization.yaml

Now the CRDs are actually being kustomized.

Signed-off-by: juliankatz <juliankatz@google.com>
First commit, adds PreserveUnknownFields: false to the non-mutation
CRDs.  The kustomization pipeline is in a bit of a weird state given it
being in alpha, so I'm not sure what else to add for that yet.

Signed-off-by: juliankatz <juliankatz@google.com>
Signed-off-by: juliankatz <juliankatz@google.com>
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: juliankatz <juliankatz@google.com>
@julianKatz julianKatz requested a review from ritazh June 22, 2021 23:28
@julianKatz julianKatz force-pushed the PreserveUnknownFields-false branch 2 times, most recently from c7a021b to 8701dad Compare June 24, 2021 19:21
… PreserveUnknownFields-false

Signed-off-by: juliankatz <juliankatz@google.com>
@julianKatz
Copy link
Contributor Author

ping :) @sozercan @ritazh @shomron

Copy link
Member

@sozercan sozercan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxsmythe maxsmythe merged commit 60c9781 into open-policy-agent:master Jun 29, 2021
julianKatz added a commit to julianKatz/gatekeeper that referenced this pull request Jul 8, 2021
* Add PreserveUnknownFields: false to CRDs

First commit, adds PreserveUnknownFields: false to the non-mutation
CRDs.  The kustomization pipeline is in a bit of a weird state given it
being in alpha, so I'm not sure what else to add for that yet.

Signed-off-by: juliankatz <juliankatz@google.com>

* Preserve --> preserve

Signed-off-by: juliankatz <juliankatz@google.com>

* Fix mutation CRDs kustomization

The patching was in the wrong place.  Per Rita's comment, I moved it
from:

config/overlays/mutation/kustomization.yaml

to

config/overlays/mutation_webhook/kustomization.yaml

Now the CRDs are actually being kustomized.

Signed-off-by: juliankatz <juliankatz@google.com>

* Add PreserveUnknownFields: false to CRDs

First commit, adds PreserveUnknownFields: false to the non-mutation
CRDs.  The kustomization pipeline is in a bit of a weird state given it
being in alpha, so I'm not sure what else to add for that yet.

Signed-off-by: juliankatz <juliankatz@google.com>

* Preserve --> preserve

Signed-off-by: juliankatz <juliankatz@google.com>

* Add load_restrictor LoadRestrictionsNone in deploy-mutation

Signed-off-by: juliankatz <juliankatz@google.com>

Co-authored-by: Sertaç Özercan <852750+sozercan@users.noreply.github.com>
Signed-off-by: juliankatz <juliankatz@google.com>
@thomasmckay
Copy link
Contributor

This fails kustomize builds on v3.6.0. Is there a fix that can be backported? (gatekeeper-operator runs kustomize against the gatekeeper release)

Raised in slack https://openpolicyagent.slack.com/archives/CDTN970AX/p1634649152271500

$ kustomize build https://github.com/open-policy-agent/gatekeeper/config/overlays/mutation_webhook/?ref=v3.6.0 -o config/gatekeeper
Error: trouble configuring builtin PatchTransformer with config: `
path: ../../crd/patches/preserve_unknown_fields_false.yaml
target:
  group: apiextensions.k8s.io
  kind: CustomResourceDefinition
  version: v1
`: security; file '/tmp/kustomize-802852871/config/crd/patches/preserve_unknown_fields_false.yaml' is not in or below '/tmp/kustomize-802852871/config/overlays/mutation_webhook'

@julianKatz
Copy link
Contributor Author

julianKatz commented Oct 20, 2021

This fails kustomize builds on v3.6.0. Is there a fix that can be backported? (gatekeeper-operator runs kustomize against the gatekeeper release)

Raised in slack https://openpolicyagent.slack.com/archives/CDTN970AX/p1634649152271500

$ kustomize build https://github.com/open-policy-agent/gatekeeper/config/overlays/mutation_webhook/?ref=v3.6.0 -o config/gatekeeper
Error: trouble configuring builtin PatchTransformer with config: `
path: ../../crd/patches/preserve_unknown_fields_false.yaml
target:
  group: apiextensions.k8s.io
  kind: CustomResourceDefinition
  version: v1
`: security; file '/tmp/kustomize-802852871/config/crd/patches/preserve_unknown_fields_false.yaml' is not in or below '/tmp/kustomize-802852871/config/overlays/mutation_webhook'

This is due to a security control that kustomize turns on by default, where it does not allow files from directories above a kustomization.yaml to be used in assembling the eventual output.

It can be disabled in your command by adding --load_restrictor LoadRestrictionsNone after the path in kustomize build.


Caveat: I tried this with the full URL style kustomize build command that you were running and it doesn't seem to work.

❯ kustomize build  'https://github.com/open-policy-agent/gatekeeper/config/overlays/mutation_webhook/?ref=v3.6.0' -o blabla.txt --load_restrictor LoadRestrictionsNone
Error: trouble configuring builtin PatchTransformer with config: `
path: ../../crd/patches/preserve_unknown_fields_false.yaml
target:
  group: apiextensions.k8s.io
  kind: CustomResourceDefinition
  version: v1
`: security; file '/tmp/kustomize-241102815/config/crd/patches/preserve_unknown_fields_false.yaml' is not in or below '/tmp/kustomize-241102815/config/overlays/mutation_webhook'

But, when I try the same thing after checking out the v3.6.0 tag you are using and referencing the directory locally, it does work:

❯ git status
HEAD detached at v3.6.0
nothing to commit, working tree clean
❯ kustomize build config/overlays/mutation_webhook -o blabla.txt --load_restrictor LoadRestrictionsNone
❯ echo $?
0

I'm using an old version of kustomize:

❯ kustomize version
{Version:kustomize/v3.8.8 GitCommit:72262c5e7135045ed51b01e417a7e72f558a22b0 BuildDate:2020-12-10T18:05:35Z GoOs:linux GoArch:amd64}

So perhaps this behavior has been fixed.

Does that unblock you? @thomasmckay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants