-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update charts to follow Helm RBAC best practices #11769
Conversation
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
|
I think it's worth adding more unit tests here to make sure everything works. Unfortunately, we are currently migrating to new framework in which writing tests will be much easier and faster. This means that we have to wait to merge this change until we have a new framework. |
Hello. We have a new test framework for Helm Chart. Can you add unit tests? Best regards, |
Sure, I'll do both of these things today. Jay |
ad0c350
to
0c3435a
Compare
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
@jaydesl What is the status of this PR? Is it ready for review? |
@mik-laj @jaydesl, I've rebased this PR with master here and added all of the additional requirements needed to get this chart deployed when working in environments where users have the PodSecurityPolicy admission controller enabled without having to provide any additional permissions to the When enabled, every pod in the cluster must have a serviceaccount associated with it that is then binded to a psp. By having the option to set the serviceaccount for every potential pod allows teams to create a single service account mapped to a psp and set that for every pod. Long term, adding psp's at a per service level (e.g. airflow, flower, pgbouncer) would be the best path forward, but this is an initial step towards supporting that. I'm happy to open up a new PR that includes this PR's changes and more, but can wait until this PR is merged and add my changes after the fact if that is deemed more appropriate. |
Related: #13643 @FloChehab Could you please make a review and say if the changes presented are in line with your issue? |
Hi @mik-laj sorry this fell off my radar. Let me know if you want me to rebase this or alternatively if its easier just go with the single PR from @DerekHeldtWerle (thanks btw!) |
Hello @mik-laj, sorry for the response delay, I had other things on top of my mind lately. The changes mentioned in the description of the PR looks like a step in the right direction, in particular in regards to #13643 & #13643 (comment). I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?). Also, there might tests to add to make sure the annotations work properly (kind of like this: https://github.com/apache/airflow/pull/13615/files). |
My PR doesn't actually add any PSP related items, all it does it create all of the necessary service accounts that are required to be set when PSP's come into play as well as rebasing with master. |
Sorry I think I saw some things related to uids and I thought there might be some more complex things related to psp in your PR. I don't really have an opinion on what is best to do. I am sure we will end up in the right spot some way or another. |
@jaydesl @FloChehab I went ahead and rebased with master again and opened up a new PR here to keep this moving along. Let me know if there's anything on the PR that needs to be addressed and I'll get to it as soon as I can 😄 |
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
This PR builds off of and supersedes @jaydesl's work on his [PR](#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: #11755 related: #13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache/airflow#11755 related: apache/airflow#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com> GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
Update the Helm charts to follow best practices around RBAC (as suggested in #11755) . Namely:
rbac.create
replacesrbacEnabled
...serviceAccount.create
...serviceAccount.name
- if not given, previous naming conventions are usedrbacEnabled = False
)closes: #11755
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.