Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update charts to follow Helm RBAC best practices #11769

Closed
wants to merge 6 commits into from

Conversation

jaydesl
Copy link
Contributor

@jaydesl jaydesl commented Oct 23, 2020

Update the Helm charts to follow best practices around RBAC (as suggested in #11755) . Namely:

  • rbac.create replaces rbacEnabled
  • ServiceAccount creation can be enabled individually with ...serviceAccount.create
  • ServiceAccounts can be named with ...serviceAccount.name - if not given, previous naming conventions are used
  • WebServer SA creation is now conditional (previously it was created even if rbacEnabled = False)

closes: #11755


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@boring-cyborg boring-cyborg bot added the area:helm-chart Airflow Helm Chart label Oct 23, 2020
@boring-cyborg
Copy link

boring-cyborg bot commented Oct 23, 2020

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, pylint and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@mik-laj
Copy link
Member

mik-laj commented Oct 23, 2020

I think it's worth adding more unit tests here to make sure everything works. Unfortunately, we are currently migrating to new framework in which writing tests will be much easier and faster. This means that we have to wait to merge this change until we have a new framework.

@mik-laj
Copy link
Member

mik-laj commented Oct 30, 2020

Hello.

We have a new test framework for Helm Chart. Can you add unit tests?

Best regards,
Kamil Breguła

@jaydesl
Copy link
Contributor Author

jaydesl commented Nov 2, 2020

Sure, I'll do both of these things today.

Jay

@jaydesl jaydesl force-pushed the master branch 2 times, most recently from ad0c350 to 0c3435a Compare November 2, 2020 11:17
@stale
Copy link

stale bot commented Dec 25, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Dec 25, 2020
@mik-laj
Copy link
Member

mik-laj commented Dec 25, 2020

@jaydesl What is the status of this PR? Is it ready for review?

@stale stale bot removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Dec 25, 2020
@DerekHeldtWerle
Copy link
Contributor

DerekHeldtWerle commented Jan 28, 2021

@mik-laj @jaydesl, I've rebased this PR with master here and added all of the additional requirements needed to get this chart deployed when working in environments where users have the PodSecurityPolicy admission controller enabled without having to provide any additional permissions to the default service account in the namespace this is deployed to.

When enabled, every pod in the cluster must have a serviceaccount associated with it that is then binded to a psp. By having the option to set the serviceaccount for every potential pod allows teams to create a single service account mapped to a psp and set that for every pod. Long term, adding psp's at a per service level (e.g. airflow, flower, pgbouncer) would be the best path forward, but this is an initial step towards supporting that.

I'm happy to open up a new PR that includes this PR's changes and more, but can wait until this PR is merged and add my changes after the fact if that is deemed more appropriate.

@mik-laj
Copy link
Member

mik-laj commented Jan 29, 2021

Related: #13643

@FloChehab Could you please make a review and say if the changes presented are in line with your issue?

@jaydesl
Copy link
Contributor Author

jaydesl commented Feb 4, 2021

Hi @mik-laj sorry this fell off my radar. Let me know if you want me to rebase this or alternatively if its easier just go with the single PR from @DerekHeldtWerle (thanks btw!)

@FloChehab
Copy link
Contributor

Related: #13643

@FloChehab Could you please make a review and say if the changes presented are in line with your issue?

Hello @mik-laj, sorry for the response delay, I had other things on top of my mind lately.

The changes mentioned in the description of the PR looks like a step in the right direction, in particular in regards to #13643 & #13643 (comment).

I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).

Also, there might tests to add to make sure the annotations work properly (kind of like this: https://github.com/apache/airflow/pull/13615/files).

@DerekHeldtWerle
Copy link
Contributor

@FloChehab

I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).

My PR doesn't actually add any PSP related items, all it does it create all of the necessary service accounts that are required to be set when PSP's come into play as well as rebasing with master.

@FloChehab
Copy link
Contributor

@FloChehab

I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).

My PR doesn't actually add any PSP related items, all it does it create all of the necessary service accounts that are required to be set when PSP's come into play as well as rebasing with master.

Sorry I think I saw some things related to uids and I thought there might be some more complex things related to psp in your PR.

I don't really have an opinion on what is best to do. I am sure we will end up in the right spot some way or another.

@DerekHeldtWerle
Copy link
Contributor

@jaydesl @FloChehab I went ahead and rebased with master again and opened up a new PR here to keep this moving along. Let me know if there's anything on the PR that needs to be addressed and I'll get to it as soon as I can 😄

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Mar 27, 2021
@github-actions github-actions bot closed this Apr 2, 2021
kaxil added a commit that referenced this pull request Apr 30, 2021
This PR builds off of and supersedes @jaydesl's work on his [PR](#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: #11755
related: #13643 

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 17, 2021
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 23, 2021
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 27, 2021
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Mar 10, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 4, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jul 9, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Aug 27, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 4, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
aglipska pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 7, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Dec 7, 2022
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jan 27, 2023
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this pull request Sep 12, 2024
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this pull request Sep 13, 2024
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 17, 2024
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 7, 2024
This PR builds off of and supersedes @jaydesl's work on his [PR](apache/airflow#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to.

closes: apache/airflow#11755
related: apache/airflow#13643

Co-authored-by: jaydesl <jay.deslauriers@gmail.com>
Co-authored-by: Ian Stanton <ian@astronomer.io>
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
GitOrigin-RevId: 8655d66cea977102862379d9894810b1e836f7a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:helm-chart Airflow Helm Chart stale Stale PRs per the .github/workflows/stale.yml policy file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RBAC and more customization in Helm Chart
4 participants