fix: Add parameters for tuning revisionHistory and securityContext

[bodgit]

What type of PR is this?

feature

Which issue does this PR fix:

n/a

What does this PR do / Why do we need it:

This PR adds support for overriding the revisionHistoryLimit setting on the Deployment as well as the setting the securityContext at both the Pod and container level.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:

Testing done on this change:

Will this PR introduce any new dependencies?:

Will this break upgrades or downgrades? Has updating a running cluster been tested?:

Defaults values are set to the implicit Kubernetes defaults.

Does this change require updates to the CNI daemonset config files to work?:

Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jdn5126]

@bodgit two questions here:

1. What is the value of the revisionHistoryLimit?
2. Why would you want a different securityContext for the pod spec and container spec where there is only one container in this pod?

[bodgit]

1. What is the value of the revisionHistoryLimit?

10 is the implicit default in current Kubernetes clusters if it's not set, so that's used as the default. My intent with the change is to be able to set this to 0 to not keep any prior revisions. Unfortunately Helm considers 0 a false value so trying to wrap the line with {{ if .Values.revisionHistoryLimit }}...{{ end }} results in no line being added for that use case.

2. Why would you want a different securityContext for the pod spec and container spec where there is only one container in this pod?

Force of habit on my part, however there's a common subset in both places but some settings can only be set in either place, (https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#securitycontext-v1-core vs https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podsecuritycontext-v1-core)

[jdn5126]

1. What is the value of the revisionHistoryLimit?

10 is the implicit default in current Kubernetes clusters if it's not set, so that's used as the default. My intent with the change is to be able to set this to 0 to not keep any prior revisions. Unfortunately Helm considers 0 a false value so trying to wrap the line with {{ if .Values.revisionHistoryLimit }}...{{ end }} results in no line being added for that use case.

2. Why would you want a different securityContext for the pod spec and container spec where there is only one container in this pod?

Force of habit on my part, however there's a common subset in both places but some settings can only be set in either place, (https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#securitycontext-v1-core vs https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#podsecuritycontext-v1-core)

Got it, thank you for explaining, and thank you for taking this up! I am fine with both of these changes. For the YAML comment, you check if the value is greater than or equal to 0, but it's not a big deal

[bodgit]

Parameter names tweaked and rebased.

[jdn5126]

Thanks @bodgit! Since you are using CNI Metrics Helper, you may also be interested in #2603, which takes the aggregated metrics published to CloudWatch and makes some of them available via Prometheus

[bodgit]

Thanks @bodgit! Since you are using CNI Metrics Helper, you may also be interested in #2603, which takes the aggregated metrics published to CloudWatch and makes some of them available via Prometheus

Yes, that does look useful. Thanks.