Bump to ansible 2.4

[joshmoore]

Migrate OME resources to Ansible 2.4. Requires updating the molecule tests in all repositories.

[joshmoore]

To get the various repos green, I'm currently testing with a fresh virtualenv that's only had pip install molecule run, giving most importantly:

$ pip freeze | grep -E ".*(ansible|molecule).*"
ansible==2.4.3.0
ansible-lint==3.4.19
molecule==2.7.0

but for completeness

$ pip freeze | grep -E ".*(ansible|molecule).*" -v
anyconfig==0.9.1
arrow==0.12.1
asn1crypto==0.24.0
attrs==17.4.0
backports.functools-lru-cache==1.4
bcrypt==3.1.4
binaryornot==0.4.4
cffi==1.11.4
chardet==3.0.4
click==6.7
click-completion==0.2.1
colorama==0.3.7
configparser==3.5.0
cookiecutter==1.5.1
cryptography==2.1.4
enum34==1.1.6
fasteners==0.14.1
flake8==3.3.0
funcsigs==1.0.2
future==0.16.0
git-url-parse==1.0.2
idna==2.6
ipaddress==1.0.19
Jinja2==2.9.6
jinja2-time==0.2.0
MarkupSafe==1.0
marshmallow==2.13.5
mccabe==0.6.1
monotonic==1.4
paramiko==2.4.0
pathspec==0.5.5
pbr==3.0.1
pexpect==4.2.1
pluggy==0.6.0
poyo==0.4.1
psutil==5.2.2
ptyprocess==0.5.2
py==1.5.2
pyasn1==0.4.2
pycodestyle==2.3.1
pycparser==2.18
pyflakes==1.5.0
PyNaCl==1.2.1
pytest==3.4.0
python-dateutil==2.6.1
python-gilt==1.1.0
PyYAML==3.12
sh==1.12.14
six==1.11.0
tabulate==0.7.7
testinfra==1.7.1
tree-format==0.1.2
whichcraft==0.4.1
yamllint==1.8.1

[joshmoore]

The circular dependency here is going to hurt. I need to bump versions in ome/ansible-role-basedeps#4 to have travis do the proper verification. Time to split the meta package from this testing package?

cc: @manics @sbesson

[sbesson]

Discussed today at the SA meeting. To some extent, we always know we need to fix this circular dependency at some point so no fundamental objection. Without too much tought it seems we could go with a structure like:

meta-package -> ome-ansible-role-{x,y} incl. individua-> testing package

Would the activation of the cron job across all individual role repos (barring the issues of automating this activation and the setup for new repos) make the downstream testing repo redundant or is it still valuable to have it typically to test an upstream dependency change across all roles?

Side comment, if we are adding shade, os_client to the meta-package, ome-ansible-molecule-dependencies is probably too specific as a name and we could even consider generalizing it name if we chose the route of extracting the meta package part into a different repo.

[manics]

👎 to including openstack modules by default in this, it brings in large number of deps. I'd support making it optional though, e.g. https://github.com/celery/celery/tree/v4.1.0#bundles

[joshmoore]

Cheers, @sbesson

[sbesson]

Following up on this, I tried to prototype the solution discussed in #10 (comment) following the examples of the recent integration repositories i.e.

• a top-level repository https://github.com/sbesson/ansible-all including all Ansible roles as well as this meta-package as submodules
• the role testing happens via Travis CI similarly to what is done in this repository - see https://travis-ci.org/sbesson/ansible-all/builds/378937988 for the passed build against master
• testing happens by merging PRs using an integration branch - see https://travis-ci.org/sbesson/ansible-all/builds/378941714 for an example including this PR
• parallely improvements have been made to scc to scale the submodule merging by automatically forking of submodules and the push order.

I propose to discuss the next steps here tomorrow at the SA meeting. A couple of initial points of discussion:

• are we generally happy with this strategy as a way forward? does this help in particular with the upgrade of the roles to Ansible 2.4?
• what is the preferred naming scheme: ansible-all? ansible-build?

[snoopycrimecop]

Conflicting PR. Removed from build ANSIBLE-merge#433. See the console output for more details.
Possible conflicts:

• Upstream changes
  • .travis.yml
  • setup.py

--conflicts Conflict resolved in build ANSIBLE-merge#435. See the console output for more details.

[joshmoore]

@sbesson : what do you want to do with this PR now that it's conflicting?

[sbesson]

@joshmoore I think I have made almost all the progress we need for resurrecting this effort:

• https://github.com/snoopycrimecop/ansible-roles/tree/master_merge_daily now is the integration branch coupling all the Ansible roles PRs including this meta-package one
• I just activated https://travis-ci.org/snoopycrimecop/ansible-roles so that coupled Ansible PRs are tested in a daily Travis build

I would propose to fix the conflict here, restrict this PR to the setup.py changes i.e. the bump to Ansible 2.4, restart the ANSIBLE-merge job and see the state of the Travis testing?

[joshmoore]

Updated as suggested, @sbesson, and travis is green.

[sbesson]

As per the Python semantics, this will install 2.4.0.0. Do we want ansible==2.4.6 or ansible<2.4?

[sbesson]

Do you rememebr why this dependency is required? I briefly looked into the state of the opened molecule v2 PRs with Ansible 2.5/2.6 following the discussion in ome/ansible-role-basedeps#5 (comment). However the initial destroy phase failed with ansible/molecule#1384. After removing docker-py from setup.py and re-installing this meta-package, as suggested in the issue and by the output of molecule --debug test, the tests passed.

[manics]

From what I remember it was a mess due to breaking changes between docker and docker-py, we definitely shouldn't have both. It'll need testing.

[joshmoore]

I don't remember what else, but docker-py is certainly critical for docker-compose integration.

[sbesson]

Indeed docker seems to be a full rewrite of docker-py and both packages certainly incompatible changes.

Looking at the requirements of docker-compose and various issues includes ansible/molecule#1347, ansible/ansible#22993:

• docker-compose nows declares docker as a dependency instead of docker-py since 2.10.0
• the Ansible Docker module used to recommend docker-py as a dependency. Recent Ansible versions seem to have included full support for docker and docker scenario guide: docker-py -> docker ansible/ansible#44856 is updating the official doc accordingly.

Switching to docker across the board sounds like the long-term option anyways. For this PR, I wonder whether the decision here is related to the targetted version of Ansible ome/ansible-role-basedeps#5 (comment). If we decide to go for Ansible 2.6, docker should work both for compose and the ansible Docker modules. If we stick to Ansible 2.4, it might be that docker-py is the best choice.

[manics]

The following combinations work on travis with idr/deployment IDR/deployment#135

• ansible==2.5.9 docker-py==1.10.6 molecule==2.18.0
• ansible==2.6.4 docker-py==1.10.6 molecule==2.18.0
• ansible==2.6.4 docker==3.5.0 molecule==2.18.0

[sbesson]

As per #10 (comment), the new proposed target is ansible 2.6/docker 3.5.0 although we might come back to this is this leads to instabilities.