refactor(deployment): refactor argo manifests to be overlay on top of upstream #5273
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Bobgy The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/assign @yanniszark @thesuperzapper I think you will be interested in the PR. |
|
/cc @zijianjoy |
There was a problem hiding this comment.
This PR seems hard to review, so I think I'll have to just make a call to merge.
Before I do that, I'd like some feedback if this seems like the right direction to go, based on PR description.
/cc @chensun @yanniszark
| # In artifactRepository.s3.endpoint, $(kfp-namespace) is needed, because in multi-user mode, pipelines may run in other namespaces. | ||
| artifactRepository: | | ||
| archiveLogs: true | ||
| s3: | ||
| endpoint: "minio-service.$(kfp-namespace):9000" | ||
| bucket: "$(kfp-artifact-bucket-name)" | ||
| keyFormat: "artifacts/{{workflow.name}}/{{pod.name}}" | ||
| # insecure will disable TLS. Primarily used for minio installs not configured with TLS | ||
| insecure: true | ||
| accessKeySecret: | ||
| name: mlpipeline-minio-artifact | ||
| key: accesskey | ||
| secretKeySecret: | ||
| name: mlpipeline-minio-artifact | ||
| key: secretkey |
There was a problem hiding this comment.
I wanted to move this patch to minio manifests, but it was more challenging than I thought.
I recorded why that's hard in #5276 and discussed some potential solutions.
|
/lgtm |
Description of your changes:
Problem: KFP maintains its own fork of argo manifests
Cons:
Proposed Solution: pull argo manifests as an upstream folder and build KFP specific argo overlays.
Challenges:
kubectl apply -k, but upstream argo manifests are not compatible with the outdated kustomize version in kubectl.Changes:
envusages in common manifests, so that they are compatible with any kustomize versionChecklist:
Do you want this pull request (PR) cherry-picked into the current release branch?
Learn more about cherry-picking updates into the release branch.