From c5b4bfb86f6b3054a7cddb41dd05af3e61bb7445 Mon Sep 17 00:00:00 2001 From: Yuan Gong Date: Thu, 26 Aug 2021 08:27:24 +0000 Subject: [PATCH 1/4] docs(kfp): 1.7 upgrade notices and breaking changes --- .../pipelines/installation/choose-executor.md | 5 +++- .../pipelines/sdk/v2/v2-compatibility.md | 20 +++------------ .../en/docs/components/pipelines/upgrade.md | 25 ++++++++++++++++--- 3 files changed, 29 insertions(+), 21 deletions(-) diff --git a/content/en/docs/components/pipelines/installation/choose-executor.md b/content/en/docs/components/pipelines/installation/choose-executor.md index 1f6b9fcd8b..1b0865f4cc 100644 --- a/content/en/docs/components/pipelines/installation/choose-executor.md +++ b/content/en/docs/components/pipelines/installation/choose-executor.md @@ -67,7 +67,10 @@ improvements can make it the default executor that most people should use in the future. Therefore, the team makes a commitment to actively collect feedback and fix bugs -for the emissary executor, so that we can stablize it faster. So far, Kubeflow +for the emissary executor, so that we can stablize it faster. +Submit your feedback in [the Emissary Executor feedback github issue](https://github.com/kubeflow/pipelines/issues/6249). + +So far, Kubeflow Pipelines test infrastructure has been running stably with the emissary executor. * Container Runtime: any diff --git a/content/en/docs/components/pipelines/sdk/v2/v2-compatibility.md b/content/en/docs/components/pipelines/sdk/v2/v2-compatibility.md index ce29a9ca29..c37d01933a 100644 --- a/content/en/docs/components/pipelines/sdk/v2/v2-compatibility.md +++ b/content/en/docs/components/pipelines/sdk/v2/v2-compatibility.md @@ -16,9 +16,9 @@ mode](http://bit.ly/kfp-v2-compatible), or join the Kubeflow Pipelines community ## Before you begin -1. Install [Kubeflow Pipelines Standalone](/docs/components/pipelines/installation/standalone-deployment) 1.7.0-rc.3 or higher. Note, support for other distributions is under development, see [Current Caveats section](#current-caveats). +1. Install [Kubeflow Pipelines Standalone](/docs/components/pipelines/installation/standalone-deployment) 1.7.0 or higher. Note, support for other distributions is under development, see [Current Caveats section](#current-caveats). -1. Run the following command to install Kubeflow Pipelines SDK v1.6.6 or higher. If you run this command in a Jupyter notebook, restart the kernel after installing the SDK. +1. Run the following command to install Kubeflow Pipelines SDK v1.7.2 or higher. If you run this command in a Jupyter notebook, restart the kernel after installing the SDK. ```bash pip install kfp --upgrade @@ -149,24 +149,10 @@ client.create_run_from_pipeline_func( ## Current Caveats -Kubeflow Pipelines v2 compatible mode is currently in Beta stage. It is under active development and some features may not be complete. Here are some of the current caveats: - -* Pipelines do not run on [the full Kubeflow] distributions with Multi-User Kubeflow Pipelines. [#5680] -* Pipelines using [v1 mlpipeline-ui-metadata visualization](/docs/components/pipelines/sdk/output-viewer/) do not compile. [#5666] -* UI integration is work-in-progress, more improvements are planned in [v2 compatible mode github project]. -* Caching is work-in-progress. [#5667] -* When consuming an input artifact by URI, its file content is still downloaded to the container regardless. [#5671] -* Pipeline components fail on non-root container images with permission problems. [#5673] +Kubeflow Pipelines v2 compatible mode is currently in Beta stage. It is under active development and some features may not be complete. To find out its current caveats, refer to [v2 compatible mode -- known caveats & breaking changes #6133](https://github.com/kubeflow/pipelines/issues/6133). [build-pipeline]: /docs/components/pipelines/sdk/v2/build-pipeline/ [build-component]: /docs/components/pipelines/sdk/v2/component-development/ [python-component]: /docs/components/pipelines/sdk/v2/python-function-components/ [kfp-client]: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client [connect-api]: /docs/components/pipelines/sdk/connect-api/ -[the full Kubeflow]: /docs/components/pipelines/installation/overview/#full-kubeflow -[v2 compatible mode github project]: https://github.com/kubeflow/pipelines/projects/13 -[#5680]: https://github.com/kubeflow/pipelines/issues/5680 -[#5666]: https://github.com/kubeflow/pipelines/issues/5666 -[#5671]: https://github.com/kubeflow/pipelines/issues/5671 -[#5673]: https://github.com/kubeflow/pipelines/issues/5673 -[#5667]: https://github.com/kubeflow/pipelines/issues/5667 diff --git a/content/en/docs/components/pipelines/upgrade.md b/content/en/docs/components/pipelines/upgrade.md index da030a2d65..380b1e6750 100644 --- a/content/en/docs/components/pipelines/upgrade.md +++ b/content/en/docs/components/pipelines/upgrade.md @@ -1,8 +1,27 @@ +++ title = "Upgrading" -description = "How to upgrade your Kubeflow Pipelines deployment" +description = "Notices and breaking changes when you upgrade Kubeflow Pipelines Backend" weight = 50 - +++ -Refer to [Upgrading Kubeflow Pipelines on Google Cloud](/docs/gke/pipelines/upgrade/). +This page introduces notices and breaking changes you need to know when upgrading Kubeflow Pipelines Backend. + +For upgrade instructions, refer to distribution specific documentations: + +* [Upgrading Kubeflow Pipelines on Google Cloud](/docs/distributions/gke/pipelines/upgrade/) + +## Upgrading to [v1.7] + +[v1.7]: https://github.com/kubeflow/pipelines/releases/tag/1.7.0 + +* **Breaking Change**: Metadata UI and visualizations are not compatible with TFX <= 1.0.0. Upgrade to TFX 1.2.0 or above, refer to [Kubeflow Pipelines Backend and TFX compatibility matrix](/docs/components/pipelines/installation/compatibility-matrix/). + +* **Notice**: Emissary executor (Alpha), a new argo workflow executor is available as an option. + + The current default docker executor does not work on Google Kubernetes Engine (GKE) 1.19+ out of the box. To use docker executor, your cluster node image must be configured to use docker (deprecated) as container runtime. + + Alternatively, using emissary executor (Alpha) removes the restriction on container runtime, but note some of your pipelines may require manual migrations. The Kubeflow Pipelines team welcomes your feedback in [the Emissary Executor feedback github issue](https://github.com/kubeflow/pipelines/issues/6249). Depending on feedback, the emissary executor may become the default in the future. + + For detailed configuration and migration instructions for both options, refer to [Argo Workflow Executors](https://www.kubeflow.org/docs/components/pipelines/installation/choose-executor/). + +* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend release, complete UI support and caching capabilities for v2 compatibility mode is newly added, [your feedback is welcomed](https://github.com/kubeflow/pipelines/issues/6451). From 084e02aa83f5341413fb1ff8865efe869cc9eb49 Mon Sep 17 00:00:00 2001 From: Yuan Gong Date: Thu, 26 Aug 2021 08:35:59 +0000 Subject: [PATCH 2/4] add a ref --- .../components/pipelines/installation/standalone-deployment.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/components/pipelines/installation/standalone-deployment.md b/content/en/docs/components/pipelines/installation/standalone-deployment.md index f975f05f5e..69807250c0 100644 --- a/content/en/docs/components/pipelines/installation/standalone-deployment.md +++ b/content/en/docs/components/pipelines/installation/standalone-deployment.md @@ -112,6 +112,8 @@ for migration instructions. ## Upgrading Kubeflow Pipelines +1. For release notices and breaking changes, refer to [Upgrading Kubeflow Pipelines](/docs/components/pipelines/upgrade). + 1. Check the [Kubeflow Pipelines GitHub repository](https://github.com/kubeflow/pipelines/releases) for available releases. 1. To upgrade to Kubeflow Pipelines 0.4.0 and higher, use the following commands: From f6b24fb724c0910b30e57df19bbeb30efb543698 Mon Sep 17 00:00:00 2001 From: Yuan Gong Date: Fri, 27 Aug 2021 04:22:31 +0000 Subject: [PATCH 3/4] address feedback --- content/en/docs/components/pipelines/upgrade.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/en/docs/components/pipelines/upgrade.md b/content/en/docs/components/pipelines/upgrade.md index 380b1e6750..d30f2e97c1 100644 --- a/content/en/docs/components/pipelines/upgrade.md +++ b/content/en/docs/components/pipelines/upgrade.md @@ -1,5 +1,5 @@ +++ -title = "Upgrading" +title = "Upgrade Notes" description = "Notices and breaking changes when you upgrade Kubeflow Pipelines Backend" weight = 50 +++ @@ -14,14 +14,14 @@ For upgrade instructions, refer to distribution specific documentations: [v1.7]: https://github.com/kubeflow/pipelines/releases/tag/1.7.0 -* **Breaking Change**: Metadata UI and visualizations are not compatible with TFX <= 1.0.0. Upgrade to TFX 1.2.0 or above, refer to [Kubeflow Pipelines Backend and TFX compatibility matrix](/docs/components/pipelines/installation/compatibility-matrix/). +* **Breaking Change**: Metadata UI and visualizations are not compatible with TensorFlow Extended (TFX) <= v1.0.0. Upgrade to v1.2.0 or above, refer to [Kubeflow Pipelines Backend and TensorFlow Extended (TFX) compatibility matrix](/docs/components/pipelines/installation/compatibility-matrix/). -* **Notice**: Emissary executor (Alpha), a new argo workflow executor is available as an option. +* **Notice**: Emissary executor (Alpha), a new argo workflow executor is available as an option. Due to [Kubernetes deprecating Docker as a container runtime after v1.20](https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/), emissary may become the default workflow executor for Kubeflow Pipelines in the near future. - The current default docker executor does not work on Google Kubernetes Engine (GKE) 1.19+ out of the box. To use docker executor, your cluster node image must be configured to use docker (deprecated) as container runtime. + For example, the current default docker executor does not work on Google Kubernetes Engine (GKE) 1.19+ out of the box. To use docker executor, your cluster node image must be configured to use docker (deprecated) as container runtime. - Alternatively, using emissary executor (Alpha) removes the restriction on container runtime, but note some of your pipelines may require manual migrations. The Kubeflow Pipelines team welcomes your feedback in [the Emissary Executor feedback github issue](https://github.com/kubeflow/pipelines/issues/6249). Depending on feedback, the emissary executor may become the default in the future. + Alternatively, using emissary executor (Alpha) removes the restriction on container runtime, but note some of your pipelines may require manual migrations. The Kubeflow Pipelines team welcomes your feedback in [the Emissary Executor feedback github issue](https://github.com/kubeflow/pipelines/issues/6249). For detailed configuration and migration instructions for both options, refer to [Argo Workflow Executors](https://www.kubeflow.org/docs/components/pipelines/installation/choose-executor/). -* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend release, complete UI support and caching capabilities for v2 compatibility mode is newly added, [your feedback is welcomed](https://github.com/kubeflow/pipelines/issues/6451). +* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend release complete UI support and caching capabilities for v2 compatibility mode are newly added, [your feedback is welcomed](https://github.com/kubeflow/pipelines/issues/6451). From 1cce2342f67db4c43eef5a8e2285770d1ce9785c Mon Sep 17 00:00:00 2001 From: Yuan Gong Date: Fri, 27 Aug 2021 04:35:11 +0000 Subject: [PATCH 4/4] move upgrade notes to installation folder --- content/en/_redirects | 7 ++++--- .../components/pipelines/{ => installation}/upgrade.md | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-) rename content/en/docs/components/pipelines/{ => installation}/upgrade.md (89%) diff --git a/content/en/_redirects b/content/en/_redirects index dbf6f808ae..d4f2679618 100644 --- a/content/en/_redirects +++ b/content/en/_redirects @@ -49,6 +49,7 @@ /docs/pipelines/sdk/reusable-components/ /docs/examples/shared-resources/ /docs/pipelines/sdk/build-component/ /docs/components/pipelines/sdk/build-pipeline/ /docs/components/pipelines/sdk/build-component/ /docs/components/pipelines/sdk/build-pipeline/ +/docs/components/pipelines/upgrade/ /docs/components/pipelines/installation/upgrade/ # Moved the guide to monitoring GKE deployments. /docs/other-guides/monitoring/ /docs/distributions/gke/monitoring/ @@ -143,7 +144,7 @@ docs/started/requirements/ /docs/started/getting-started/ /docs/started/workstation/minikf-aws /docs/distributions/minikf/minikf-aws /docs/started/workstation/minikf-gcp /docs/distributions/minikf/minikf-gcp -# Distinguish components from external add-ons +# Distinguish components from external add-ons /docs/components/fairing /docs/external-add-ons/fairing /docs/components/istio /docs/external-add-ons/istio /docs/components/feature-store /docs/external-add-ons/feature-store @@ -154,7 +155,7 @@ docs/started/requirements/ /docs/started/getting-started/ /docs/started/getting-started /docs/started/installing-kubeflow # =============== -# IMPORTANT NOTE: +# IMPORTANT NOTE: # Catch-all redirects should be added at the end of this file as redirects happen from top to bottom # =============== /docs/guides/* /docs/:splat @@ -168,4 +169,4 @@ docs/started/requirements/ /docs/started/getting-started/ /docs/components/fairing/* /docs/external-add-ons/fairing/:splat /docs/components/istio/* /docs/external-add-ons/istio/:splat /docs/components/feature-store/* /docs/external-add-ons/feature-store/:splat -/docs/components/serving/* /docs/external-add-ons/serving/:splat \ No newline at end of file +/docs/components/serving/* /docs/external-add-ons/serving/:splat diff --git a/content/en/docs/components/pipelines/upgrade.md b/content/en/docs/components/pipelines/installation/upgrade.md similarity index 89% rename from content/en/docs/components/pipelines/upgrade.md rename to content/en/docs/components/pipelines/installation/upgrade.md index d30f2e97c1..6f6e2dd316 100644 --- a/content/en/docs/components/pipelines/upgrade.md +++ b/content/en/docs/components/pipelines/installation/upgrade.md @@ -1,7 +1,7 @@ +++ title = "Upgrade Notes" description = "Notices and breaking changes when you upgrade Kubeflow Pipelines Backend" -weight = 50 +weight = 35 +++ This page introduces notices and breaking changes you need to know when upgrading Kubeflow Pipelines Backend. @@ -24,4 +24,4 @@ For upgrade instructions, refer to distribution specific documentations: For detailed configuration and migration instructions for both options, refer to [Argo Workflow Executors](https://www.kubeflow.org/docs/components/pipelines/installation/choose-executor/). -* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend release complete UI support and caching capabilities for v2 compatibility mode are newly added, [your feedback is welcomed](https://github.com/kubeflow/pipelines/issues/6451). +* **Notice**: [Kubeflow Pipelines SDK v2 compatibility mode](/docs/components/pipelines/sdk/v2/v2-compatibility/) (Beta) was recently released. The new mode adds support for tracking pipeline runs and artifacts using ML Metadata. In v1.7 backend, complete UI support and caching capabilities for v2 compatibility mode are newly added, [your feedback is welcomed](https://github.com/kubeflow/pipelines/issues/6451).