Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make KFP v2 compatible mode work out of the box #284

Merged

Conversation

Bobgy
Copy link
Contributor

@Bobgy Bobgy commented May 28, 2021

Part of kubeflow/pipelines#5680

Depends on kubeflow/pipelines#5750

Changes:

  • configure default pipeline root to gcs bucket using a new kpt setter default-pipeline-root
  • extract KFP specific pull-upstream script to its own folder, so that admin can only update KFP and deploy

@google-oss-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bobgy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Bobgy
Copy link
Contributor Author

Bobgy commented May 28, 2021

/hold
wait for upstream PR to merge first

@Bobgy
Copy link
Contributor Author

Bobgy commented May 28, 2021

/ok-to-test

export KUBEFLOW_PIPELINES_VERSION=1.5.0
export KUBEFLOW_PIPELINES_REPO=https://github.com/kubeflow/pipelines.git
# Pull Kubeflow Pipelines upstream manifests.
./apps/pipelines/pull-upstream.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use make -C apps/pipelines pull-upstream instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point. I am not sure. If we add it to makefile the model is more consistent with other commands.

If use separate pull-upstream.sh, packages without makefile can easily add them too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I ask is that you have already created a pull-upstream make target in https://github.com/kubeflow/gcp-blueprints/blob/2b8e52dd10796d78e40f40d6d9617b9e64b773b3/kubeflow/apps/pipelines/Makefile#L73, so I guess the goal is to use it.

As mentioned in the other comment, if the eventual state is to separate pull-upstream commands to each sub-folder, or use kpt subpackage when it becomes available, I prefer to create a template script for pull-upstream, so each sub-folder can call this template from Makefile, instead of having pull-upstream.sh in each sub-folder. Because it will increase amount of files in the whole repo, and it is hard to keep them consistent. If there is no Makefile, then we will use the default template to pull upstream.

For this PR, feel free to keep the current usage. And we can update them at one go when we break down the kubeflow/pull-upstream.sh.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I added the makefile rule for consistency.

If there is no Makefile, then we will use the default template to pull upstream.

I think we need to know where upstream is and which version, these info need to be somewhere in the folder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logged in #287

kubeflow/kpt-set.sh Show resolved Hide resolved
kubeflow/apps/pipelines/pull-upstream.sh Show resolved Hide resolved
@zijianjoy
Copy link
Collaborator

/lgtm

Waiting for upstream target to be changed to KFP 1.6.1.

@Ark-kun
Copy link

Ark-kun commented Jun 2, 2021

Perhaps we can rename this PR "make KFP v2 compatible mode work out of the box".

@Bobgy Bobgy changed the title feat: make KFP v2 compatible mode run by default feat: make KFP v2 compatible mode run out of the box Jun 2, 2021
@Bobgy Bobgy changed the title feat: make KFP v2 compatible mode run out of the box feat: make KFP v2 compatible mode work out of the box Jun 2, 2021
@Bobgy
Copy link
Contributor Author

Bobgy commented Jun 4, 2021

/hold
still need to wait

@zijianjoy
Copy link
Collaborator

Any plan for merging this PR?

@zijianjoy zijianjoy mentioned this pull request Sep 4, 2021
16 tasks
@Bobgy
Copy link
Contributor Author

Bobgy commented Sep 7, 2021

/unhold
Yes, this is ready for merge.

@google-oss-robot google-oss-robot merged commit c292668 into GoogleCloudPlatform:master Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants