Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unsupported Media Type (HTTP 415) when patching custom resources #866

Closed
nolar opened this issue Jul 4, 2019 · 9 comments
Closed

Unsupported Media Type (HTTP 415) when patching custom resources #866

nolar opened this issue Jul 4, 2019 · 9 comments
Assignees

Comments

@nolar
Copy link

nolar commented Jul 4, 2019

Actual behaviour

v10.0.0 was released this night. Since then, custom resource patching is broken:

HTTP 415 "Unsupported Media Type" when patching the custom resources.

Traceback (most recent call last):
  File "kopf-134.py", line 15, in <module>
    rsp = api.patch_namespaced_custom_object(**request_kwargs)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/apis/custom_objects_api.py", line 1951, in patch_namespaced_custom_object
    (data) = self.patch_namespaced_custom_object_with_http_info(group, version, namespace, plural, name, body, **kwargs)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/apis/custom_objects_api.py", line 2057, in patch_namespaced_custom_object_with_http_info
    collection_formats=collection_formats)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 334, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 168, in __call_api
    _request_timeout=_request_timeout)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/api_client.py", line 393, in request
    body=body)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/rest.py", line 286, in PATCH
    body=body)
  File "/Users/svasilyev/.pyenv/versions/kopf/lib/python3.7/site-packages/kubernetes/client/rest.py", line 222, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (415)
Reason: Unsupported Media Type
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Thu, 04 Jul 2019 11:53:38 GMT', 'Content-Length': '263'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the body of the request was in an unknown format - accepted media types include: application/json-patch+json, application/merge-patch+json","reason":"UnsupportedMediaType","code":415}

When tracing step by step, the actual Content-Type used is application/json-patch+json, which should match the supported types from the error message.

There is no way to control Content-Type from the user side, i.e. when calling the patch-method.

Expected behaviour

Patching works smoothly.

It all works fine if installed as

pip install 'kubernetes<10.0.0'

Steps to reproduce:

minikube start --kubernetes-version=v1.15.0
kubectl apply -f https://raw.githubusercontent.com/zalando-incubator/kopf/master/examples/crd.yaml
kubectl apply -f https://raw.githubusercontent.com/zalando-incubator/kopf/master/examples/obj.yaml

Any other CRD should show the same behaviour (probably); this one is just for a quick example.

pip install kubernetes==10.0.0

Then a Python script:

import kubernetes

kubernetes.config.load_kube_config()  # developer's config files

request_kwargs = {
    'group': 'zalando.org',
    'version': 'v1',
    'plural': 'kopfexamples',
    'name': 'kopf-example-1',
    'namespace': 'default',
    'body': {},
}

api = kubernetes.client.CustomObjectsApi()
rsp = api.patch_namespaced_custom_object(**request_kwargs)
print(rsp)

Any body content cause the same error, event the empty body — so, I guess, it is not about the patch-content itself.

Versions

Kubernetes: 1.15.0
kubernetes==10.0.0
Python 3.7

Notes

Might be related:

@nolar
Copy link
Author

nolar commented Jul 4, 2019

Found the root cause:

With 9.0.0, the type was application/merge-patch+json.
With 10.0.0, it is application/json-patch+json.

The select_header_content_type() method choses the first content type from a list of available. The content is not checked:

            return content_types[0]

The CustomObjectsApi feeds it with these 2 content-types:

        header_params['Content-Type'] = self.api_client.\
            select_header_content_type(['application/json-patch+json', 'application/merge-patch+json'])

Previously (in 9.0.0), it was one:

        header_params['Content-Type'] = self.api_client.\
            select_header_content_type(['application/merge-patch+json'])

This happened because of 9c8bd4a.

And so, the default content type has suddenly changed since 9.0 to 10.0.0.

The Kubernetes API doesn't like it, because application/json-patch+json is a list, not dict (see Wikipedia):

[
    { "op": "add", "path": "/myPath", "value": ["myValue"] }
]

However, the content data sent by the users all over the world was not changed, and most likely remains a dict as per application/merge-patch+json content-type (since it worked before).

PS: With body changed from {} to [], the example from the issue description works.

@roycaihw
Copy link
Member

roycaihw commented Jul 9, 2019

/assign

@roycaihw
Copy link
Member

roycaihw commented Jul 17, 2019

Thanks for the detailed bug report. I see the select_header_content_type is not intelligent as I would expected.

kube-apiserver supports both JSON patch and JSON merge patch as content types for patching custom resources. The problem is select_header_content_type unconditionally puts the first type from the list into the HTTP header. So it used to be people using the python client can only use JSON patch as body (which was bad), but upgrading the python client suddenly changes the HTTP header to use JSON merge patch, which breaks existing users. What's more, JSON patch is no longer an option given how select_header_content_type works.

I think we should do two things:

@roycaihw
Copy link
Member

fixed in 10.0.1

/close

@k8s-ci-robot
Copy link
Contributor

@roycaihw: Closing this issue.

In response to this:

fixed in 10.0.1

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@abdulsalama
Copy link

which version is this fix in?

I still get the same problem when patching profiles.
kubectl patch profile foo --patch ....

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3"
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.9-eks-502bfb",

Any idea why I'm still seeing it?

palnabarun added a commit to palnabarun/python that referenced this issue Jun 22, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Jun 22, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Jul 16, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Aug 19, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Oct 12, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Oct 14, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 7, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 7, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 13, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 13, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 13, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 16, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 16, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 16, 2020
palnabarun added a commit to palnabarun/python that referenced this issue Nov 16, 2020
roycaihw pushed a commit to roycaihw/client-python that referenced this issue Apr 11, 2021
roycaihw pushed a commit to roycaihw/client-python that referenced this issue Apr 12, 2021
@ableuler
Copy link

ableuler commented Jun 1, 2021

  • For long-term, we should give users the option to specify the content-type. It can be defaulted as JSON patch, but currently it's really hard for users to hijack the HTTP request and change the header. I think removing select_header_content_type is part of kubernetes-client/gen#93. Ref #738 (comment)

This would be great!

Until this is implemented and for everybody coming here in search for a way to apply a JSON patch to a custom resource with content_type="application/json-patch+json", the dynamic client supports this option already. See for example here.

@retr0h
Copy link

retr0h commented Nov 17, 2021

Reading this issue, I still don't understand what needs done to actually use this ;)

I'm calling the following code with version 19.15.0 of the python bindings.

        body = [{"op": "remove", "path": "/spec/remoteWrite"}]
        response = api.patch_namespaced_custom_object(
            group="monitoring.coreos.com",                                                                                                                  version="v1",
            name="kube-prometheus-stack-prometheus",
            namespace="monitoring",                                                                                                                         plural="prometheuses",
            body=body,

However, I receive the following error:

Reason: Unprocessable Entity
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '95e7d775-0b04-4674-bcba-f35bf7a9cac8', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'aa6ddeef-d18c-429c-a54e-d2be86b818b6', 'Date': 'Wed, 17 Nov 2021 01:33:09 GMT', 'Content-Length': '812'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":" \"\" is invalid: patch: Invalid value: \"[{\\\"op\\\":\\\"remove\\\",\\\"path\\\":\\\"/spec/remoteWrite\\\"}]\": couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }","reason":"Invalid","details":{"causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"[{\\\"op\\\":\\\"remove\\\",\\\"path\\\":\\\"/spec/remoteWrite\\\"}]\": couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }","field":"patch"}]},"code":422}

/cc @roycaihw @nolar

@jmgilman
Copy link

jmgilman commented Mar 2, 2023

For future readers, the body is a JSON patch which is described here. In particular, it took me a while to figure out how to add an annotation with a / in it:

        body = [
            {
                "op": "add",
                # '~1' is an escaped '/'
                # See https://jsonpatch.com/#json-pointer for escaping rules
                "path": "/metadata/annotations/myproject.io~1archive-name",
                "value": archive.metadata.name,
            }
        ]

bh717 pushed a commit to bh717/python-dapp that referenced this issue Apr 1, 2024
Reference:
- [#866](kubernetes-client/python#866)
- [#959](kubernetes-client/python#959)

Signed-off-by: Nabarun Pal <pal.nabarun95@gmail.com>
bh717 pushed a commit to bh717/python-dapp that referenced this issue Apr 1, 2024
Reference:
- [#866](kubernetes-client/python#866)
- [#959](kubernetes-client/python#959)

Signed-off-by: Nabarun Pal <pal.nabarun95@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants