-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add distribution of K8s hooks #995
Conversation
Another approach I thought about was insteads of sites adding scripts to their pre-hook.sh , we make them opt in via the configuration they already have to supply in |
…ns are handled by hook.env. Reduces duplicate scripting logic and ensures fewer executions of kubectl
What role does hook.env play? FWIW I know right now that the Lua code sends the hook arguments to STDIN of the webhook, but I would still like to explore having it do what Jeff first attempted, which was actually set the environment prior to executing the script. If we switch to that, does that have any impact on this PR? Also I notice most of the files are under hooks/k8s-bootstrap/ but some are just under hooks/ - would it be better if all K8s related hook scripts and data where namespaced under the k8s-bootstrap directory? I imagine in the future we would add here other example hooks for things unrelated to k8s? |
The If the nginx_stage could set all these environment variables then we would not need hook.env but they would all have to be set and some of them are considered secret and should only be accessible by root so they should not make their way into user's PUN environment, like the I organized the scripts to live under |
I can imagine one possible way to handle hook env is a new key/value pair in nginx_stage like:
Which then sets like |
0eb358a
to
6b94c63
Compare
Just to hone the discussion a bit, here's an outline of the 3 current possibilities.
/opt/ood/hooks/k8s/apply_one_thing.sh
/opt/ood/hooks/k8s/apply_some_other_thing.sh
/opt/ood/hooks/k8s/apply.sh /opt/ood/hooks/config/k8s/user_namespace.yaml
/opt/ood/hooks/k8s/apply.sh /opt/ood/hooks/config/k8s/pod_security_policy.yaml
For my taste, I'd vote for number 2. It allows for a lot of composability where a site could mix and match site specific things with what we ship. It's also more explicit as a pose to number 3. The admin can directly see what's being applied and how. K8s is bound to generate a lot of tickets and discussion so being explicit about what's being set and how may help out some user confusion. |
What is the status of this PR? Is it urgent to merge this today? |
I think it's still a bit in flux, I don't think it's urgent enough to need today. |
I think it's needed sometime before we announce 2.0 as officially released but with enough time before that for OSC to test the changes and deploy them. What this PR does is replicated in osc-ood-configs repo so for OSC there is no urgency other than to ensure we can migrate to use what's in this PR and test it. |
This will fix #994 .
I left things so there are 2 options show. Large inline string that is dumped into tmp file and applied or YAML generated using envsubts and dumped into tmp files and then applied. A site could choose to add the pod-security-policy or job-pod-reaper executions to their pre-hook.sh if they wanted those opt-in items.
I added what is an attempt to validate the K8s YAML using real Kubernetes cluster from kind cluster action.