Add extend CLI for generate policies proposal #6

shubham4443 · 2022-03-23T19:24:14Z

Signed-off-by: Shubham Nazare shubham4443@gmail.com

realshuting

Had a few questions for advanced support:

can we handle context lookups for generate policies using CLI? (Is this supported in validate and mutate policies via CLI?)
can we handle JMESPath variables? (Is this supported in mutate and validate?)
can we use existing resources in the cluster for this CLI test?
how do we handle policies with spec.rules.generate.synchronize set to true?

realshuting · 2022-03-24T06:21:47Z

proposals/extend_cli_for_generate_policies.md

+
+The method is similar to testing mutate policies. User will provide the desired resource manifest file as generatedResource in results array of the test file. This manifest file will be compared with the Kyverno generated resource and accordingly the test will display the result as pass or fail.
+
+Additionally, like in the case of testing mutate policies, we provide the resource on which we want to perform mutation. For generate policies, the triggering resource should be provided on creation/updation of which our generation rule is applied.


..provided on creation/update of which..

realshuting · 2022-03-24T06:25:36Z

proposals/extend_cli_for_generate_policies.md

+
+### Mock Generate-Request Controller
+
+To process generate rules, we need a Generate-Request Controller. We can make a simple instance of the Controller with only the Kubernetes' dynamic client. The dynamic client must be a fake one so as to not process the rule on an actual cluster. Additionally, the triggering resource and pre-existing resource for the clone object should exist in the cluster. The dynamic client will be initialized with these resources.


Based on the above proposal, all test resources are local resources, do we need the dynamic client?

realshuting · 2022-03-24T06:25:37Z

proposals/extend_cli_for_generate_policies.md

+
+### Mock Generate-Request Controller
+
+To process generate rules, we need a Generate-Request Controller. We can make a simple instance of the Controller with only the Kubernetes' dynamic client. The dynamic client must be a fake one so as to not process the rule on an actual cluster. Additionally, the triggering resource and pre-existing resource for the clone object should exist in the cluster. The dynamic client will be initialized with these resources.


Based on the above proposal, all test resources are local resources, do we need the dynamic client?

Yes we require the dynamic client otherwise its not possible to invoke the function responsible for handling generate policies.

realshuting

Had a few questions for advanced support:

can we handle context lookups for generate policies using CLI? (Is this supported in validate and mutate policies via CLI?)
can we handle JMESPath variables? (Is this supported in mutate and validate?)
can we use existing resources in the cluster for this CLI test?
how do we handle policies with spec.rules.generate.synchronize set to true?

chipzoller · 2022-03-24T16:41:34Z

One of the objectives of this enhancement would be to ensure all the presently-available generate policies on kyverno/policies can be tested successfully.

proposals/extend_cli_for_generate_policies.md

vyankyGH · 2022-04-05T06:13:27Z

@shubham4443 Have we updated document for Namespace based generate policies ?

shubham4443 · 2022-05-30T11:50:11Z

Hi @chipzoller @realshuting
kyverno/kyverno#3456 has been merged. Is there anything left in this design proposal?

chipzoller · 2022-05-30T13:08:59Z

Hi @chipzoller @realshuting kyverno/kyverno#3456 has been merged. Is there anything left in this design proposal?

There are comments asked by Shuting which you have not addressed. At this point, since we didn't really follow the process we created, let's just make sure this KDP reflects the current state of whatever was implemented. In other words, if there are things in this KDP which are different based on the current state, go back and update the KDP accordingly.

shubham4443 · 2022-05-30T13:21:32Z

Hi @chipzoller @realshuting kyverno/kyverno#3456 has been merged. Is there anything left in this design proposal?

There are comments asked by Shuting which you have not addressed. At this point, since we didn't really follow the process we created, let's just make sure this KDP reflects the current state of whatever was implemented. In other words, if there are things in this KDP which are different based on the current state, go back and update the KDP accordingly.

I addressed those comments in our weekly contributor's meeting. Nevertheless, I will answer those questions again. I have gone through the KDP just now and there aren't any things different from the current state.

shubham4443 · 2022-05-30T13:25:38Z

Had a few questions for advanced support:

can we handle context lookups for generate policies using CLI? (Is this supported in validate and mutate policies via CLI?)

can we handle JMESPath variables? (Is this supported in mutate and validate?)

can we use existing resources in the cluster for this CLI test?

how do we handle policies with spec.rules.generate.synchronize set to true?

Answers to the questions -

Yes - We can handle context lookups for generate policies and it is supported in validate and mutate policies too.
Yes - We can handle JMESPath variables and it is supported in validate and mutate policies too.
No
We are not handling policies with spec.rules.generate.synchronize set to true any differently.

chipzoller · 2022-05-30T14:06:39Z

@realshuting to provide final review and merge.

realshuting

@shubham4443 - looks good overall.

Have a question regarding the name, see below

realshuting · 2022-05-31T13:01:41Z

proposals/extend_cli_for_generate_policies.md

+
+## Proposal
+
+When using a generate rule, the origin resource can be either an existing resource defined within Kubernetes, or a new resource defined in the rule itself. When the origin resource is a pre-existing resource the clone object is used. When the origin resource is a new resource defined within the manifest of the rule, the data object is used. Both rules are mutually exclusive, and only one will be specified in a rule. Both objects require a different approach for testing through CLI command.


We usually have the trigger and the target resources in a generate policy. What is this origin resource?

Its the same as target resource. In the documentation https://kyverno.io/docs/writing-policies/generate/ it is mentioned as origin resource so I used the same word.

Can we rename them to the target resource? Ideally we should be consistent when referencing trigger and target resources for generate and mutate existing policies.

realshuting · 2022-05-31T13:03:43Z

proposals/extend_cli_for_generate_policies.md

+
+### Method to test data object
+
+The method is similar to testing mutate policies. User will provide the desired resource manifest file as generatedResource in the results array of the test file. This manifest file will be compared with the Kyverno generated resource and accordingly the test will display the result as pass or fail.


.. is similar to the testing for mutate policies.

realshuting · 2022-05-31T13:06:03Z

proposals/extend_cli_for_generate_policies.md

+    result: pass
+```
+
+User should provide the pre-existing resource in the cloneSourceResource field of the results array. The cloneSourceResource.yaml for the above test file will look like this -


.. in the cloneSourceResource field ..

realshuting · 2022-06-01T07:09:30Z

@shubham4443 - can you resolve conversations that were addressed and sign off your commits?

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Signed-off-by: shubham4443 <shubham4443@gmail.com>

Signed-off-by: Naman Lakhwani <namanlakhwani@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Signed-off-by: shubham4443 <shubham4443@gmail.com>

vyankyGH · 2022-06-01T16:16:25Z

@shubham4443 please resolves conflicts

…_for_generate_policies

realshuting · 2022-06-01T16:58:14Z

Was the implementation PR merged? May need to the change status on main page.

shubham4443 · 2022-06-01T17:02:13Z

Was the implementation PR merged? May need to the change status on main page.

Yes

realshuting reviewed Mar 24, 2022

View reviewed changes

chipzoller reviewed Mar 24, 2022

View reviewed changes

proposals/extend_cli_for_generate_policies.md Outdated Show resolved Hide resolved

chipzoller requested a review from realshuting May 30, 2022 14:06

realshuting reviewed May 31, 2022

View reviewed changes

shubham4443 and others added 13 commits June 1, 2022 12:55

Add extend CLI for generate policies proposal

d3c6515

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Fix some grammar mistakes

840f8c9

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Fix grammar and add API Design Changes

0f32087

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Change cloneResource to cloneSourceResource

e192f66

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Fix indentation

4a975dd

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Add details for namespaced policy

6b9d33d

Signed-off-by: Shubham Nazare <shubham4443@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Fix typo mistakes

4e2af5e

Signed-off-by: shubham4443 <shubham4443@gmail.com>

correcting hyperlinks

048ac73

Signed-off-by: Naman Lakhwani <namanlakhwani@gmail.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

update digest tags

567453f

Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

add table

b17242a

Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

fix link

2b0ee17

Signed-off-by: Jim Bugwadia <jim@nirmata.com> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Add KDP review/acceptance procedure + template (kyverno#12)

89685ca

Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net> Signed-off-by: shubham4443 <shubham4443@gmail.com>

Rename origin to target

691f028

Signed-off-by: shubham4443 <shubham4443@gmail.com>

shubham4443 force-pushed the extend_cli_for_generate_policies branch from 6e20ff8 to 691f028 Compare June 1, 2022 07:25

Merge branch 'main' of https://github.com/kyverno/KDP into extend_cli…

db38d60

…_for_generate_policies

vyankyGH approved these changes Jun 1, 2022

View reviewed changes

realshuting approved these changes Jun 1, 2022

View reviewed changes

chipzoller merged commit 0b764a1 into kyverno:main Jun 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add extend CLI for generate policies proposal #6

Add extend CLI for generate policies proposal #6

shubham4443 commented Mar 23, 2022

realshuting left a comment

realshuting Mar 24, 2022

realshuting Mar 24, 2022

realshuting Mar 24, 2022

shubham4443 May 30, 2022

realshuting left a comment

chipzoller commented Mar 24, 2022

vyankyGH commented Apr 5, 2022

shubham4443 commented May 30, 2022

chipzoller commented May 30, 2022

shubham4443 commented May 30, 2022

shubham4443 commented May 30, 2022

chipzoller commented May 30, 2022

realshuting left a comment

realshuting May 31, 2022

shubham4443 Jun 1, 2022

realshuting Jun 1, 2022

realshuting May 31, 2022

realshuting May 31, 2022

realshuting commented Jun 1, 2022

vyankyGH commented Jun 1, 2022

realshuting commented Jun 1, 2022

shubham4443 commented Jun 1, 2022


		The method is similar to testing mutate policies. User will provide the desired resource manifest file as generatedResource in results array of the test file. This manifest file will be compared with the Kyverno generated resource and accordingly the test will display the result as pass or fail.

		Additionally, like in the case of testing mutate policies, we provide the resource on which we want to perform mutation. For generate policies, the triggering resource should be provided on creation/updation of which our generation rule is applied.


		### Mock Generate-Request Controller

		To process generate rules, we need a Generate-Request Controller. We can make a simple instance of the Controller with only the Kubernetes' dynamic client. The dynamic client must be a fake one so as to not process the rule on an actual cluster. Additionally, the triggering resource and pre-existing resource for the clone object should exist in the cluster. The dynamic client will be initialized with these resources.


		## Proposal

		When using a generate rule, the origin resource can be either an existing resource defined within Kubernetes, or a new resource defined in the rule itself. When the origin resource is a pre-existing resource the clone object is used. When the origin resource is a new resource defined within the manifest of the rule, the data object is used. Both rules are mutually exclusive, and only one will be specified in a rule. Both objects require a different approach for testing through CLI command.


		### Method to test data object

		The method is similar to testing mutate policies. User will provide the desired resource manifest file as generatedResource in the results array of the test file. This manifest file will be compared with the Kyverno generated resource and accordingly the test will display the result as pass or fail.

Add extend CLI for generate policies proposal #6

Add extend CLI for generate policies proposal #6

Conversation

shubham4443 commented Mar 23, 2022

realshuting left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

realshuting left a comment

Choose a reason for hiding this comment

chipzoller commented Mar 24, 2022

vyankyGH commented Apr 5, 2022

shubham4443 commented May 30, 2022

chipzoller commented May 30, 2022

shubham4443 commented May 30, 2022

shubham4443 commented May 30, 2022

chipzoller commented May 30, 2022

realshuting left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

realshuting commented Jun 1, 2022

vyankyGH commented Jun 1, 2022

realshuting commented Jun 1, 2022

shubham4443 commented Jun 1, 2022