Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680. Fixes #5704 #5750

Merged

Conversation

Bobgy
Copy link
Contributor

@Bobgy Bobgy commented May 28, 2021

Description of your changes:
Part of #5680, Fixes #5704

The field defaultPipelineRoot field in pipeline-install-config can be used to configure both multi-user mode and KFP standalone.

This PR depends on feature in #5697.

Checklist:

@Bobgy
Copy link
Contributor Author

Bobgy commented May 28, 2021

/assign @zijianjoy @capri-xiyue

@Bobgy Bobgy changed the title feat(deployment): configurable v2 compatible mode default pipeline root feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680 May 28, 2021
@Bobgy
Copy link
Contributor Author

Bobgy commented May 28, 2021

/hold
wait for #5697 to merge first

@Bobgy Bobgy changed the title feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680 feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680, Fixes #5704 May 28, 2021
@Bobgy Bobgy changed the title feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680, Fixes #5704 feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680, Part of #5704 May 28, 2021
@Bobgy Bobgy changed the title feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680, Part of #5704 feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680 and #5704 May 28, 2021
@Bobgy Bobgy changed the title feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680 and #5704 feat(deployment): configurable v2 compatible mode default pipeline root. Part of #5680. Fixes #5704 May 28, 2021
## When not in Kubeflow Pipelines multi-user mode, the config works as you
## would normally expect.
##
## In Kubeflow Pipelines multi-user mode, the config creates default
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do all namespaces share same default pipeline root path as specified in defaultPipelineRoot field under multi-user mode? How do we control data access if all namespaces share same pipeline root?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users can edit the kfp-launcher configmap afterwards and their changes won't be overridden again by this config.

Users can configure different defaultPipelineRoot per namespace in kfp-launcher configmap. Do you have any suggestions to make this part easier to understand?

let me try to clarify in the PR first

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Yuan for the doc update! I understand it better now. I was originally thinking that how do we programmatically assign a pipeline root for each namespace? For example: gs://{$BUCKET_NAME}/v2/artifact/{$NAMESPACE}. But it seems to be out-of-scope for current version, because we need to edit kfp-launcher-configmap.yaml in each user's namespace.

Also understand it is hard to configure pipeline root based on namespace parameter, the current solution SGTM.

@Bobgy
Copy link
Contributor Author

Bobgy commented Jun 2, 2021

@zijianjoy What do you think about the updated documentation?

@Bobgy
Copy link
Contributor Author

Bobgy commented Jun 2, 2021

/unhold
/approve
#5697 has been merged

@Bobgy
Copy link
Contributor Author

Bobgy commented Jun 2, 2021

/unhold
/approve
#5697 has been merged

Copy link
Collaborator

@zijianjoy zijianjoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Thank you Yuan for the change!

## When not in Kubeflow Pipelines multi-user mode, the config works as you
## would normally expect.
##
## In Kubeflow Pipelines multi-user mode, the config creates default
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Yuan for the doc update! I understand it better now. I was originally thinking that how do we programmatically assign a pipeline root for each namespace? For example: gs://{$BUCKET_NAME}/v2/artifact/{$NAMESPACE}. But it seems to be out-of-scope for current version, because we need to edit kfp-launcher-configmap.yaml in each user's namespace.

Also understand it is hard to configure pipeline root based on namespace parameter, the current solution SGTM.

@google-oss-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bobgy, zijianjoy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-robot google-oss-robot merged commit adc1951 into kubeflow:master Jun 2, 2021
@Bobgy Bobgy deleted the config-default-pipeline-root branch June 3, 2021 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[v2compat] configurable default pipeline root
4 participants