Add Docker provider #977

wainersm · 2019-02-25T15:59:13Z

This set of commits add basic docker provider which is able to manage container and image resources.

On commit 79ad73c for the docker_image resource, it was used the path_relative_to filter to make some parameters relative to the workspace. Same thing could be implemented with some parameters of the docker_container resource, however I will wait the review of this set of commits before I follow up with that.

It was not implemented test cases because I still don't understand how they can be implemented. I appreciate if anyone gives some guidance on it. It could come on a follow up patch as well...

Fixes #957

adl-bot · 2019-02-25T17:30:27Z

Can one of the admins verify this patch?

p3ck · 2019-02-26T18:26:51Z

I think this needs block: sections around the docker calls in roles/docker/tasks/provision_docker_* with code added for installing docker dependencies like we do for the other providers.

TASK [docker : Provision Docker image] *****************************************************************
task path: /home/bpeck/Projects/LinchPin/lib/python2.7/site-packages/linchpin/provision/roles/docker/tasks/provision_docker_image.yml:16
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "api_version": "auto", 
            "archive_path": null, 
            "buildargs": null, 
            "cacert_path": null, 
            "cert_path": null, 
            "container_limits": null, 
            "debug": false, 
            "docker_host": "unix://var/run/docker.sock", 
            "dockerfile": "fedora29", 
            "force": false, 
            "http_timeout": null, 
            "key_path": null, 
            "load_path": null, 
            "name": "custom_fedora29", 
            "nocache": false, 
            "path": "/home/bpeck/Sandbox/contra/linchpin/docs/source/examples/workspaces/docker/dockerfiles", 
            "pull": true, 
            "push": false, 
            "repository": null, 
            "rm": true, 
            "ssl_version": null, 
            "state": "present", 
            "tag": "python", 
            "timeout": 60, 
            "tls": false, 
            "tls_hostname": "localhost", 
            "tls_verify": false, 
            "use_tls": "no"
        }
    }, 
    "msg": "Failed to import docker or docker-py - No module named docker. Try `pip install docker` or `pip install docker-py` (Python 2.6)"
}
	to retry, use: --limit @/home/bpeck/Projects/LinchPin/lib/python2.7/site-packages/linchpin/provision/docker.retry

PLAY RECAP *********************************************************************************************
localhost                  : ok=9    changed=0    unreachable=0    failed=1   

Unsuccessful provision of resource

wainersm · 2019-02-27T18:39:37Z

I think this needs block: sections around the docker calls in roles/docker/tasks/provision_docker_* with code added for installing docker dependencies like we do for the other providers.

I just realized linchpin has a "linchpin setup " command. Btw, opened a PR to improve it #993

Taken provision/roles/libvirt/tasks/provision_libvirt_node.yml as an example, I should:

Enclose the ansible's docker module call in a block/rescue, and the message tells the user to run "linchpin setup docker"
Add a playbook under roles/dependencies/tasks/ for docker installation
Edit roles/dependencies/tasks/main.yml properly

@p3ck , on regarding dependencies installation, that's all I should do?

p3ck · 2019-02-27T18:46:23Z

@wainersm sounds correct. I'll re-review when you post here. Thanks!

adl-bot · 2019-02-28T00:45:48Z

ci-linchpin-messageBus-trigger build #59120: FAILURE: https://jenkins-continuous-infra.cloud.continuous-infra.upshift.redhat.com/job/ci-linchpin-messageBus-trigger/59120/

wainersm · 2019-02-28T12:30:34Z

@p3ck Updated the PR, the setup instructions are isolated in a commit.

p3ck · 2019-02-28T15:12:18Z

@wainersm Thank you. We are in the middle of releasing 1.7.2 so will look to merge this after that is finished.

wainersm · 2019-03-11T17:34:11Z

@p3ck I've just pushed the commit e6f6c23 that introduces tests to the docker provider. I tests passed on my Fedora 19 machine, where I did:

cd config/Dockerfiles
make all
./run_tests.sh $LINCHPIN_SRC fedora29
make DISTROS=fedora28 all
./run_tests.sh $LINCHPIN_SRC fedora28

Note: 1) changed run_tests.sh to execute docker provider tests 2) failed to run on centos7, but actually it did not installed linchpin on the container so I think it is not related with my changes.

wainersm · 2019-03-11T17:58:04Z

Rebased the code to latest so that it resolved a conflict in config/Dockerfiles/tests.d/inventory/01_template_inventory file.

adl-bot · 2019-03-11T18:40:59Z

ci-linchpin-messageBus-trigger build #59158: FAILURE: https://jenkins-continuous-infra.cloud.continuous-infra.upshift.redhat.com/job/ci-linchpin-messageBus-trigger/59158/

adl-bot · 2019-03-11T19:27:02Z

ci-linchpin-messageBus-trigger build #59159: FAILURE: https://jenkins-continuous-infra.cloud.continuous-infra.upshift.redhat.com/job/ci-linchpin-messageBus-trigger/59159/

wainersm · 2019-03-20T14:38:13Z

@samvarankashyap is needed something from my side to move this PR forward?

14rcole · 2019-04-02T14:34:55Z

@wainersm can you rebase against develop? Thanks!

samvarankashyap · 2019-04-18T13:24:13Z

moving it to 1.7.5 since we are waiting on rebase

wainersm · 2019-04-23T21:27:32Z

@14rcole @samvarankashyap I've just rebased it to latest.

14rcole · 2019-04-24T12:57:17Z

@wainersm looks like you've got some unit test/formatting errors

samvarankashyap · 2019-04-24T16:17:58Z

@wainersm Please fix the following flake8 errors:

$ flake8 --exclude=\.eggs,tests,docs --ignore=E124,E303,W504 --max-line-length 80 .
./linchpin/provision/filter_plugins/path_relative_to.py:5:1: E302 expected 2 blank lines, found 1
./linchpin/provision/filter_plugins/path_relative_to.py:10:28: E231 missing whitespace after ','
./linchpin/provision/filter_plugins/path_relative_to.py:10:33: E231 missing whitespace after ','
./linchpin/provision/filter_plugins/path_relative_to.py:10:39: E231 missing whitespace after ','
./linchpin/provision/filter_plugins/path_relative_to.py:14:1: E302 expected 2 blank lines, found 1
./linchpin/provision/filter_plugins/path_relative_to.py:17:17: E126 continuation line over-indented for hanging indent
./linchpin/provision/filter_plugins/path_relative_to.py:18:17: E123 closing bracket does not match indentation of opening bracket's line
./linchpin/provision/filter_plugins/path_relative_to.py:19:1: W391 blank line at end of file
./linchpin/InventoryFilters/DockerInventory.py:8:1: E302 expected 2 blank lines, found 1
./linchpin/InventoryFilters/DockerInventory.py:34:1: W391 blank line at end of file
The command "flake8 --exclude=\.eggs,tests,docs --ignore=E124,E303,W504 --max-line-length 80 ." exited with 1.

The docker provider introduced on this change allows one to provision a docker container. This implementation is a wrapper around Ansible's docker_container module. Fixes issue CentOS-PaaS-SIG#957

This filter expect a base_path and path arguments, check if path is absolute or relative to the OS filesystem then: 1. In either case, return the path as is. 2. Otherwise, return the base_path + path. Examples: -> {{ '/path/to/files' | path_relative_to('/path/to/workspace') }} <- "/path/to/files" -> {{ 'files' | path_relative_to('/path/to/workspace') }} <- "/path/to/workspace/files"

This extend the docker provider to allow creation/destroy of container images. Currently it is a wrapper around the Ansible's docker_image module, but it does not allow to push or archive an image.

Added support for installing docker dependencies with `linchpin setup docker`

Added tests for docker setup and container provisioning. It was disabled execution of general/01_general and inventory/01_template_inventory because they required docker setup correctly in the test container.

wainersm · 2019-04-24T18:49:14Z

@samvarankashyap @14rcole ok, now I fixed all checks fail. :)

samvarankashyap · 2019-04-29T15:24:18Z

@wainersm I see all check pass now, We will try to incorporate docker provider into 1.7.5 release.
Thank you for your contribution.

wainersm · 2019-05-08T12:27:40Z

@samvarankashyap hello! any update on this PR?

14rcole · 2019-05-08T12:42:03Z

@wainersm now that 1.7.4 has been released, we can merge this

14rcole · 2019-05-08T12:42:11Z

[merge]

wainersm · 2019-05-09T14:29:07Z

Thanks @14rcole !

samvarankashyap requested review from 14rcole, samvarankashyap and p3ck February 25, 2019 16:10

samvarankashyap added docker rfe provider labels Feb 25, 2019

samvarankashyap added this to the v1.7.3 milestone Feb 25, 2019

samvarankashyap modified the milestones: v1.7.3, v1.7.2 Feb 25, 2019

p3ck modified the milestones: v1.7.2, v1.7.3 Feb 26, 2019

wainersm force-pushed the docker_provider_v3 branch from 79ad73c to 5fae0b4 Compare February 27, 2019 23:57

wainersm force-pushed the docker_provider_v3 branch from 5fae0b4 to e6f6c23 Compare March 11, 2019 17:26

wainersm force-pushed the docker_provider_v3 branch from e6f6c23 to 820a6e8 Compare March 11, 2019 17:56

samvarankashyap modified the milestones: v1.7.3, v1.7.4 Mar 19, 2019

samvarankashyap modified the milestones: v1.7.4, 1.7.5 Apr 18, 2019

wainersm force-pushed the docker_provider_v3 branch from 820a6e8 to 8a7c2ac Compare April 23, 2019 21:26

samvarankashyap added the community label Apr 24, 2019

wainersm added 5 commits April 24, 2019 12:51

Add docker provider

75647b0

The docker provider introduced on this change allows one to provision a docker container. This implementation is a wrapper around Ansible's docker_container module. Fixes issue CentOS-PaaS-SIG#957

docker: provision container image

2f2000c

This extend the docker provider to allow creation/destroy of container images. Currently it is a wrapper around the Ansible's docker_image module, but it does not allow to push or archive an image.

docker: add setup support

00755d9

Added support for installing docker dependencies with `linchpin setup docker`

docker: add tests

116d869

Added tests for docker setup and container provisioning. It was disabled execution of general/01_general and inventory/01_template_inventory because they required docker setup correctly in the test container.

wainersm force-pushed the docker_provider_v3 branch from 8a7c2ac to 116d869 Compare April 24, 2019 17:06

14rcole approved these changes May 8, 2019

View reviewed changes

adl-bot merged commit f3f1b1b into CentOS-PaaS-SIG:develop May 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Docker provider #977

Add Docker provider #977

wainersm commented Feb 25, 2019

adl-bot commented Feb 25, 2019

p3ck commented Feb 26, 2019

wainersm commented Feb 27, 2019

p3ck commented Feb 27, 2019

adl-bot commented Feb 28, 2019

wainersm commented Feb 28, 2019

p3ck commented Feb 28, 2019

wainersm commented Mar 11, 2019 •

edited

Loading

wainersm commented Mar 11, 2019

adl-bot commented Mar 11, 2019

adl-bot commented Mar 11, 2019

wainersm commented Mar 20, 2019

14rcole commented Apr 2, 2019

samvarankashyap commented Apr 18, 2019

wainersm commented Apr 23, 2019

14rcole commented Apr 24, 2019

samvarankashyap commented Apr 24, 2019

wainersm commented Apr 24, 2019

samvarankashyap commented Apr 29, 2019

wainersm commented May 8, 2019

14rcole commented May 8, 2019

14rcole commented May 8, 2019

wainersm commented May 9, 2019

Add Docker provider #977

Add Docker provider #977

Conversation

wainersm commented Feb 25, 2019

adl-bot commented Feb 25, 2019

p3ck commented Feb 26, 2019

wainersm commented Feb 27, 2019

p3ck commented Feb 27, 2019

adl-bot commented Feb 28, 2019

wainersm commented Feb 28, 2019

p3ck commented Feb 28, 2019

wainersm commented Mar 11, 2019 • edited Loading

wainersm commented Mar 11, 2019

adl-bot commented Mar 11, 2019

adl-bot commented Mar 11, 2019

wainersm commented Mar 20, 2019

14rcole commented Apr 2, 2019

samvarankashyap commented Apr 18, 2019

wainersm commented Apr 23, 2019

14rcole commented Apr 24, 2019

samvarankashyap commented Apr 24, 2019

wainersm commented Apr 24, 2019

samvarankashyap commented Apr 29, 2019

wainersm commented May 8, 2019

14rcole commented May 8, 2019

14rcole commented May 8, 2019

wainersm commented May 9, 2019

wainersm commented Mar 11, 2019 •

edited

Loading