-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
urllib3 v2 incompatibility #3113
Comments
I think also the request version needs to be pinned, as there was a change?
|
I can confirm just pinning |
That's because older versions of |
I think urllib3 2.0.0 (or vendored urllib3 verions in requests) is a red herring; the problem is requests 2.29.0 itself because of a new feature: https://github.com/psf/requests/pull/6226/files Docker SDK for Python uses requests with its own HTTP adapters for Unix connections by default (if you don't talk to the Docker daemon over TCP connections). For that, the code in Docker SDK for Python that provides a Unix connection HTTP adapter needs to be adjusted to support the same chunking support that urllib3 (already < 2.0.0) provides, and that requests 2.29.0 uses. (The other HTTP adapters might have the same problems.) |
Pins back requests to avoid docker/docker-py#3113
Wow this post is really helpful as I've just encounter an issue with Ansible managing Docker. |
I created a PR for making Docker SDK for Python compatible with requests 2.29.0: #3116. It fixes the problems for me (as long as I stick to urllib3 < 2.0, what request has been requiring for a long time now). It seems to fix the problems I had with requests 2.29.0 and urllib3 < 2.0; would be nice to have more real life testing (especially for all the transports I don't use). |
Hello, urllib3 2.0 maintainer here 👋 First, let me mention that right now, if you use I can confirm however that docker-py is not compatible with urllib3 2.0. Since requests will soon allow it, docker-py should modify setup.py to use So what is the issue? Here is how docker-py supports Unix domain sockets: docker-py/docker/transport/unixconn.py Lines 17 to 37 in a02ba74
It assumes that I've opened #3117 for a fix. |
@pquentin as you mention installing docker with a recent pip will give you requests 2.29.0 and urllib3 < 2.0, so urllib3 2.0 shouldn't be a problem in that case. But the stack trace printed above isn't the only thing that can go wrong with requests 2.29.0, I found some problems in CI with requests 2.29.0 and urllib3 1.26.15 as well. These happen because requests 2.29.0 includes https://github.com/psf/requests/pull/6226/files, which ends up calling BTW, your fix only fixes one of the transports; my similar PR (#3116) catches some more. |
Docker is installed over pip with the dependencies:
And we have the same problem: The only solution for us was: pip3 remove requests |
…lib3 >=2 Upstream issue: docker/docker-py#3113
Hello from downstream consumer of And what do you mean by "recent enough" pip? We hit this issue on Ubuntu 20.04 with pip-22.0.4. |
I meant at least pip 20.3, and ideally 21.3 or above, to get the new resolver and all its fixes. Unfortunately the recent enough pip is no longer enough as requests 2.30.0 was just released with urllib3 2.0 support. Until docker-py pins urllib3 or merges #3116, you’ll have to pin to urllib3<2.0 yourself. As far as I can tell this has nothing to do with requests, so you can use the latest here. |
You can run the following with from docker import APIClient
from docker.utils import kwargs_from_env
kwargs = kwargs_from_env()
client = APIClient(**kwargs)
def generate_data():
yield b'123'
container = client.create_container(image='debian:bullseye', command=['/bin/sh', '-c', 'ls -la /'], stdin_open=True, detach=True, name='test')
client.put_archive(container['Id'], '/', generate_data()) Stacktrace:
So yes, you need |
This is a temporary workaround for docker/docker-py#3113
|
This was fixed in community.docker 3.4.5, are you using an older version? Please report any further issues you have to https://github.com/ansible-collections/community.docker instead. |
I was, yes. I'd also tried switching between the O/S (Ubuntu 22.04) package and PIP installs of docker-py on both host and target, and getting nowhere, but hadn't through to update the Ansible collection, which has resolved the issue! |
See: docker/docker-py#3113 For more details
In case someone lands on this issue like I did trying to use ansible to manage Docker on a Ubuntu 24 LTS server, a quick solution. Make sure to use the new "community.docker.docker_compose_v2" task instead of the v1 "community.docker.docker_compose" task. You get this exact same error with v1. |
Thanks. I spend hours lookinng for a solution for installing Octoprint docker and your comment was the only good suggestion. |
The issue docker/docker-py#3113 seems to be resolved. The pinned version of requests (2.28.1) is no longer compatible with the current docker pip, resulting in errors like: "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker" This commit allows pip to install requests according to its dependency tree. Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The issue docker/docker-py#3113 seems to be resolved. The pinned version of requests (2.28.1) is no longer compatible with the current docker pip, resulting in errors like: "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker" This commit allows pip to install requests according to its dependency tree. Fixes: confidential-containers#381 Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The issue docker/docker-py#3113 seems to be resolved. The pinned version of requests (2.28.1) is no longer compatible with the current docker pip, resulting in errors like: "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker" This commit allows pip to install requests according to its dependency tree. Fixes: confidential-containers#381 Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The issue docker/docker-py#3113 seems to be resolved. The pinned version of requests (2.28.1) is no longer compatible with the current docker pip, resulting in errors like: "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker" Firstly, we try to allow pip to install requests according to its dependency tree. But, the issue still occurs on Ubuntu 22.04 where docker/docker-py#3256 looks relevant. The best solution so far is to pin the version of the package less than 2.32. Initially, we attempted to allow pip to install requests according to its dependency tree. However, the issue persists on Ubuntu 22.04, and it appears to be related to docker/docker-py#3256. The best solution so far is to pin the version of the package to less than 2.32. Fixes: confidential-containers#381 Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
The issue docker/docker-py#3113 seems to be resolved. The pinned version of requests (2.28.1) is no longer compatible with the current docker pip, resulting in errors like: "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker" Initially, we attempted to allow pip to install requests according to its dependency tree. However, the issue persists on Ubuntu 22.04, and it appears to be related to docker/docker-py#3256. The best solution so far is to pin the version of the package to less than 2.32. Fixes: confidential-containers#381 Signed-off-by: Hyounggyu Choi <Hyounggyu.Choi@ibm.com>
Older version of docker-py is affected by docker/docker-py#3113
Older version of docker-py is affected by docker/docker-py#3113
By inheriting from `urllib3.connection.HTTPConnection` (that inherits from `httplib.HTTPConnection` itself), we can adapt to the internal changes in urllib3 2.0 that added a `request()` method that is incompatible with httplib.HTTPConnection.request. This fixes the incompatibility between urllib3 2.0 and requests 1.26+, which was the first version that stopped vendoring urllib3. Reference: docker/docker-py#3113 (comment) Fixes: #70
Older version of docker-py is affected by docker/docker-py#3113
Older version of docker-py is affected by docker/docker-py#3113
Minimal repro:
urllib3 2.0.0 just released today. A quick fix would be to pin to
urllib3<2
.The text was updated successfully, but these errors were encountered: