docker-py installation breaks docker-compose

[serialdoom]

im not quite sure if this is correct, but trying to install docker-py through pip after i've installed docker-compose breaks docker-compose with

Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 7, in <module>
    from compose.cli.main import main
  File "/usr/local/lib/python2.7/site-packages/compose/cli/main.py", line 20, in <module>
    from ..bundle import get_image_digests
  File "/usr/local/lib/python2.7/site-packages/compose/bundle.py", line 13, in <module>
    from .network import get_network_defs_for_service
  File "/usr/local/lib/python2.7/site-packages/compose/network.py", line 7, in <module>
    from docker.types import IPAMConfig
ImportError: cannot import name IPAMConfig

To fix that error, i just need to do the installations in this order:

pip install docker-py
pip install docker-compose

gist:
https://gist.github.com/serialdoom/3a443c420aa29f9422f8c5fc73f46602

python/pip versions tried:

docker run -it python:2.7.13 bash -c 'pip --version'
pip 9.0.1 from /usr/local/lib/python2.7/site-packages (python 2.7)
docker run -it python:2.7.12 bash -c 'pip --version'
pip 8.1.2 from /usr/local/lib/python2.7/site-packages (python 2.7)

[aboutlo]

I have the very same problem, I rolled back to 1.10.5 to fix the issue

[serialdoom]

apparently my proposed fix doesnt quite work as its breaking docker-py.

instead of @aboutlo sulution, downgrating docker-compose to 1.9.0 also fixes the problem

[shin-]

The newer version of docker-compose uses the docker package which unfortunately conflicts with the docker-py package. I'd suggest using virtualenv to isolate those projects, or make sure to only use docker (not docker-py) going forward.

HTH.

[aboutlo]

thank you @shin- for the explanation.
Does it mean we can use docker-compose without docker-py?

Currently, I'm installing docker via an ansible role. And as you can see there is an open issue there as well angstwad/docker.ubuntu#135

So if we could rid off docker-py it could be simpler to have a proper fix.

[aboutlo]

I double check here: https://github.com/docker/compose/blob/master/requirements.txt
docker-py doesn't seem to me a docker-compose dependency.

So I'm not getting how to install the last versions of docker docker-compose docker-py into a linux box.

[shin-]

@aboutlo You do not need docker-py for the most recent version of docker-compose. It has been replaced by the docker package.

If you absolutely MUST have docker-compose and docker-py in the same environment, your best bet is probably to use the binary version of docker-compose here

[james-stephenson]

For those coming here because they are investigating this issue as a result of using ansible, you can no longer install docker-compose via pip and also use the ansible docker module together, which requires docker-py.

Our workaround was to not use the ansible docker module (unfortunately), and instead use docker directly.

[shin-]

@james-stephenson

Our workaround was to not use the ansible docker module (unfortunately), and instead use docker directly.

Is there any reason you chose to do that over using the docker-compose binary, or using virtualenv to separate docker-compose and ansible? I'm mostly curious as this seems very radical.

[james-stephenson]

The specific case in which this failed was just due to pulling docker images onto the machine during our AMI generation. This use case is rather specific to how we're using ansible. ansible hadn't supported a docker pull module in earlier versions, and if I recall correctly there were also some issues with ansible's Docker module incorrectly comparing versions lexically rather than numerically (I would have to dig that PR up to remember why).

Needless to say, we've had a few issues here and there with ansible's docker support, so we had the docker pull code ready to go as a remediation from previous PRs. It ends up being no more complex than using the docker module, so for us it doesn't actually add any tech debt despite the fact that it may sound radical.

We are entertaining the idea of using the docker-compose binary, but it's actually easier to manage the installation of docker-compose using pip right now in our ansible configuration. Hope that helps clarify our use case and why we went this route for the time being.

[shin-]

Thank you for taking the time to clarify!

[isaacweathers]

Old but fixed for this issue:

1. pip install --upgrade pip
2. pip install -e 'git+https://github.com/shin-/compose.git@1.6.2#egg=docker-compose'
3. pip uninstall docker-py
4. sudo pip install docker-py

This seemed to work for me on Mac.