Fresh install renders 'no matches for kind "Application"' #1933

pdmack · 2018-11-09T18:12:16Z

I've seen this twice but it seems to remedy itself by regenerating and reapplying the ks app.

+ ks apply default -c application
INFO Applying customresourcedefinitions applications.app.k8s.io
INFO Creating non-existent customresourcedefinitions applications.app.k8s.io
ERROR handle object: patching object from cluster: merging object with existing state: unable to recognize "/tmp/ksonnet-mergepatch884886323": no matches for kind "Application" in version "app.k8s.io/v1beta1"

The text was updated successfully, but these errors were encountered:

kkasravi · 2018-11-10T04:26:44Z

Thanks Pete. It sounds like the crd isn't getting registered or possibly the crd is getting registered with the wrong scope. Is this on master?

pdmack · 2018-11-10T15:16:38Z

Yes

jlewi · 2018-12-27T03:03:01Z

@kkasravi @pdmack any idea whether this is still happening?

cheyang · 2019-01-06T08:12:39Z

I'm able to reproduce it in branch v0.4-branch. Will we fix this issue before v0.4 release?

${KUBEFLOW_SRC}/scripts/kfctl.sh apply k8s
+ ks apply default -c application
INFO Updating customresourcedefinitions applications.app.k8s.io
INFO  Creating non-existent customresourcedefinitions applications.app.k8s.io
ERROR handle object: merging object with existing state: unable to recognize "/tmp/ksonnet-mergepatch060322179": no matches for kind "Application" in version "app.k8s.io/v1beta1"

xuchenCN · 2019-01-16T03:07:39Z

Hi @cheyang I encountered the same issue , but its disppear when I re-apply the ksonnet , have some idea for that ?
BTW , may I ask what do you do to overcome the GFW to pull the gcr.io images ?

kkasravi · 2019-01-17T06:31:02Z

you should try

      ks show default -c metacontroller -c application > default.yaml
      kubectl apply --validate=false -f default.yaml

application has a dependency on metacontroller. This is what kfctl.sh is doing on master.

cheyang · 2019-01-19T11:36:58Z

Hi @cheyang I encountered the same issue , but its disppear when I re-apply the ksonnet , have some idea for that ?
BTW , may I ask what do you do to overcome the GFW to pull the gcr.io images ?

You can run this on Alibaba Cloud by following the document below which is in Chinese: https://yq.aliyun.com/articles/686672

jlewi · 2019-02-03T22:25:45Z

Has anyone found a way to reproduce this?

13567436138 · 2019-02-04T07:56:19Z

when i apply ks show default -c metacontroller -c application > default.yaml
I got ks show default -c metacontroller -c application > default.yaml
what‘s going on？

13567436138 · 2019-02-04T07:56:27Z

got ERROR finding app root from starting path: : unable to find ksonnet project

jlewi · 2019-03-02T18:19:27Z

I observed this error. The logs clearly indicate the apps resource CR was created.

customresourcedefinition.apiextensions.k8s.io "applications.app.k8s.io" created
...
INFO|2019-03-02T17:57:06|/mnt/test-data-volume/auto_deploy/testing/py/kubeflow/testing/util.py|71| unable to recognize "default.yaml": no matches for kind "Application" in version "app.k8s.io/v1beta1"

pods.txt

I wonder if it takes some time for the CRD to be loaded. Does simply retrying help?

jlewi · 2019-03-02T18:53:20Z

It looks like retrying fixes things.

Below are the logs the first time kubectl apply --validate=false -f default.yaml it fails with the error about the application kind even though the custom resource was created. Rerunning the kubectl apply --validate=false -f default.yaml it succeeds.

logs.txt

@Branch

* Explicitly delete the storage deployment; the delete script won't delete it by design because we don't want to destroy data. * Instead of depending on a dummy kubeconfig file call generate/apply for platform and then for k8s. * For repos take them in the form ${ORG}/${NAME}@Branch. This matches what the other test script does. It also allows us to check out the repos from people's forks which makes testing changes easier. * Move logic in checkout-snapshot.sh into repo_clone_snapshot.py This is cleaner then having the python script shell out to a shell script. * Move the logic in deployment-workflow.sh into create_kf_instance.py * Add an option in create_kf_instance.py to read and parse the snapshot.json file rather than doing it in bash. * Change the arg name to be datadir instead of nfs_mount because nfs is an assumption of how it is run on K8s. * Check out the source into NFS to make it easier to debug. * Add a bash script to set the images using YQ * Add a wait operation for deletes and use it to wait for deletion of storage. * Rename init.sh to auto_deploy.sh to make it more descriptive. * Also modify auto_deploy.sh so we can run it locally. * Use the existing worker ci image as a base image to deduplicate the Dockerfile. * Attach labels to the deployment not the cluster * We want to use deployments not cluster to decide what to recycle * Deloyments are global but clusters are zonal and we want to be able to move to different zones to deal with issues like stockouts. * The GCP API does return labels on deployments. * We can figure out which deployment to recycle just by looking at the insertTime; we don't need to depend on deployment labels. * Add retries to deal with kubeflow/kubeflow#1933

@Branch

* Improve auto_deploy to support changing zone and testing changes. * Explicitly delete the storage deployment; the delete script won't delete it by design because we don't want to destroy data. * Instead of depending on a dummy kubeconfig file call generate/apply for platform and then for k8s. * For repos take them in the form ${ORG}/${NAME}@Branch. This matches what the other test script does. It also allows us to check out the repos from people's forks which makes testing changes easier. * Move logic in checkout-snapshot.sh into repo_clone_snapshot.py This is cleaner then having the python script shell out to a shell script. * Move the logic in deployment-workflow.sh into create_kf_instance.py * Add an option in create_kf_instance.py to read and parse the snapshot.json file rather than doing it in bash. * Change the arg name to be datadir instead of nfs_mount because nfs is an assumption of how it is run on K8s. * Check out the source into NFS to make it easier to debug. * Add a bash script to set the images using YQ * Add a wait operation for deletes and use it to wait for deletion of storage. * Rename init.sh to auto_deploy.sh to make it more descriptive. * Also modify auto_deploy.sh so we can run it locally. * Use the existing worker ci image as a base image to deduplicate the Dockerfile. * Attach labels to the deployment not the cluster * We want to use deployments not cluster to decide what to recycle * Deloyments are global but clusters are zonal and we want to be able to move to different zones to deal with issues like stockouts. * The GCP API does return labels on deployments. * We can figure out which deployment to recycle just by looking at the insertTime; we don't need to depend on deployment labels. * Add retries to deal with kubeflow/kubeflow#1933 * Fix lint. * * With K8s 1.11 we need to set volumeName otherwise we get storage class not found. Related to kubeflow/kubeflow#2475 * Fix lint. * * Change cron job to run every 12 hours. * This should be the current schedule but it looks like it was never checked in * We want to leave clusters up long enough to facilitate debugging.

andrewcheny · 2019-03-20T01:12:00Z

I have exactly the same error on the first run (fresh install). I run the script for the 2nd time and the problem is fixed.

jlewi · 2019-03-31T23:28:44Z

I wonder if there is a race condition and we are trying to create an App resource before the App CRD is fully ready?

kkasravi · 2019-04-01T19:29:22Z

since we removed the application component (in lieu of the kubernetes-sigs/application controller) I don't think this can occur anymore. Closing.

kkasravi · 2019-04-01T19:29:27Z

/close

k8s-ci-robot · 2019-04-01T19:29:28Z

@kkasravi: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jonasdebeukelaer · 2020-08-26T08:57:44Z

Hey reopening to say this error still popped up for me on a 1.1 install, but at a different point:

service/application-controller-service created
serviceaccount/application-controller-service-account created
error: unable to recognize ".build/application/app.k8s.io_v1beta1_application_application-controller-kubeflow.yaml": no matches for kind "Application" in version "app.k8s.io/v1beta1"
make: *** [apply-kubeflow] Error 1

rerunning make apply does fix it as before.

also getting it for kind Image:

service/controller created
service/webhook created
serviceaccount/controller created
error: unable to recognize ".build/knative/caching.internal.knative.dev_v1alpha1_image_queue-proxy.yaml": no matches for kind "Image" in version "caching.internal.knative.dev/v1alpha1"
make: *** [apply-kubeflow] Error 1

and Profile:

serviceaccount/pytorch-operator created
serviceaccount/tf-job-dashboard created
serviceaccount/tf-job-operator created
error: unable to recognize ".build/kubeflow-apps/kubeflow.org_v1beta1_profile_default-profile.yaml": no matches for kind "Profile" in version "kubeflow.org/v1beta1"
make: *** [apply-kubeflow] Error 1

and here too rerunning the command works.

issue-label-bot · 2020-08-26T08:57:51Z

Issue-Label Bot is automatically applying the labels:

Label	Probability
kind/bug	0.77

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

everestas · 2022-09-22T13:40:37Z

I have the same error when deploying on AKS. To deploy I'm using this command:

export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_k8s_istio.v1.2.0.yaml"
kfctl apply -V -f ${CONFIG_URI}

I am getting this error:

WARN[0077] Encountered error applying application application:  (kubeflow.error): Code 500 with message: Apply.Run : 
[unable to recognize "/tmp/kout234490513": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1", unable to recognize "/tmp/kout234490513": no matches for kind "Application" in version "app.k8s.io/v1beta1"]  filename="kustomize/kustomize.go:284"

I've tried rerunning, but the same errors keeps popping up.

Juandavi1 · 2022-11-19T00:52:01Z

same here

obeyda · 2022-12-02T19:02:48Z

having the same issue on AKS with kfctl_k8s_istio.v1.2.0.yaml

jlewi added the area/bootstrap label Dec 27, 2018

jlewi mentioned this issue Feb 4, 2019

I am installing kubeflow and got this error，no matches for kind "Application" in version "app.k8s.io/v1beta1"，need help #2380

Closed

jlewi mentioned this issue Mar 2, 2019

Improve auto_deploy to support changing zone and testing changes. kubeflow/testing#323

Merged

k8s-ci-robot closed this as completed Apr 1, 2019

JasonLee925 mentioned this issue Apr 3, 2019

install-kubeflow.sh fails with error "merging object with existing state: unable to recognize ..." #2545

Closed

nrchakradhar mentioned this issue Oct 3, 2019

kfctl#36 Split build and deploy to separate tests #4187

Merged

issue-label-bot bot added the kind/bug label Aug 26, 2020

wkeifenheim mentioned this issue Mar 23, 2021

Kubeflow 1.2 GCP install - Istio pods stuck in containerCreating #5714

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fresh install renders 'no matches for kind "Application"' #1933

Fresh install renders 'no matches for kind "Application"' #1933

pdmack commented Nov 9, 2018

kkasravi commented Nov 10, 2018

pdmack commented Nov 10, 2018

jlewi commented Dec 27, 2018

cheyang commented Jan 6, 2019 •

edited

Loading

xuchenCN commented Jan 16, 2019 •

edited

Loading

kkasravi commented Jan 17, 2019 •

edited

Loading

cheyang commented Jan 19, 2019 •

edited

Loading

jlewi commented Feb 3, 2019

13567436138 commented Feb 4, 2019

13567436138 commented Feb 4, 2019

jlewi commented Mar 2, 2019

jlewi commented Mar 2, 2019

andrewcheny commented Mar 20, 2019

jlewi commented Mar 31, 2019

kkasravi commented Apr 1, 2019

kkasravi commented Apr 1, 2019

k8s-ci-robot commented Apr 1, 2019

jonasdebeukelaer commented Aug 26, 2020

issue-label-bot bot commented Aug 26, 2020

everestas commented Sep 22, 2022 •

edited

Loading

Juandavi1 commented Nov 19, 2022

obeyda commented Dec 2, 2022

Fresh install renders 'no matches for kind "Application"' #1933

Fresh install renders 'no matches for kind "Application"' #1933

Comments

pdmack commented Nov 9, 2018

kkasravi commented Nov 10, 2018

pdmack commented Nov 10, 2018

jlewi commented Dec 27, 2018

cheyang commented Jan 6, 2019 • edited Loading

xuchenCN commented Jan 16, 2019 • edited Loading

kkasravi commented Jan 17, 2019 • edited Loading

cheyang commented Jan 19, 2019 • edited Loading

jlewi commented Feb 3, 2019

13567436138 commented Feb 4, 2019

13567436138 commented Feb 4, 2019

jlewi commented Mar 2, 2019

jlewi commented Mar 2, 2019

andrewcheny commented Mar 20, 2019

jlewi commented Mar 31, 2019

kkasravi commented Apr 1, 2019

kkasravi commented Apr 1, 2019

k8s-ci-robot commented Apr 1, 2019

jonasdebeukelaer commented Aug 26, 2020

issue-label-bot bot commented Aug 26, 2020

everestas commented Sep 22, 2022 • edited Loading

Juandavi1 commented Nov 19, 2022

obeyda commented Dec 2, 2022

cheyang commented Jan 6, 2019 •

edited

Loading

xuchenCN commented Jan 16, 2019 •

edited

Loading

kkasravi commented Jan 17, 2019 •

edited

Loading

cheyang commented Jan 19, 2019 •

edited

Loading

everestas commented Sep 22, 2022 •

edited

Loading