Reuse beats' docker-compose files in the PoC

[mdelapenya]

We want a way to get the latest docker compose files until this is merged into the Beats repo

Because supported versions are hardcoded in the PoC (see Gherkin tables) we have to think about a manner to keep those versions in sync with what comes from the supported versions file from an integration. We'd like to avoid the overwork of updating the feature files each time a version appears/disappears/is updated.

[mdelapenya]

Idea: Now that docker-compose files are in sync with Beats, we can create a command in the CLI that fetches beats repo and copies integrations compose files into the tool workspace. Something like:

$ op sync integrations

[mdelapenya]

@jsoriano I noticed that in a few compose files from Beats, there are different environment variables not based on integration's name, i.e.:

• dropwizard: MAVEN_VERSION
• uwsgi: PYTHON_VERSION

Besides that, NATS contains two services in its compose:

version: '2.3'

services:
  nats:
    image: docker.elastic.co/integrations-ci/beats-nats:${NATS_VERSION:-2.0.4}-1
    build:
      context: ./_meta
      dockerfile: Dockerfile.2.0.X
      args:
        NATS_VERSION: ${NATS_VERSION:-2.0.4}
    ports:
      - 8222

  nats_1_3:
    image: docker.elastic.co/integrations-ci/beats-nats:${NATS_VERSION:-1.3.0}-1
    build:
      context:./_meta
      dockerfile: Dockerfile.1.3
      args:
        NATS_VERSION: ${NATS_VERSION:-1.3.0}
    ports:
      - 8222

if we pass NATS_VERSION, then both services will be using it, even though the Dockerfile is using different paths and binaries as entrypoint. Is this intended or a bug?

Also uwsgi contains two services, one for TCP and the other for HTTP traffic. What will happen if we add supported_versions in them? The build/push process on CI will override the first one with the second.

[jsoriano]

@jsoriano I noticed that in a few compose files from Beats, there are different environment variables not based on integration's name, i.e.:

We should probably review them, but we don't have features by now for these services, could we continue without changing them by now?

* dropwizard: MAVEN_VERSION

We could hardcode this version, and actually parameterize the dropwizard version that we are hardcoding in the pom.xml file, and is the thing that we are interested to test.

* uwsgi: PYTHON_VERSION

Same thing, we could hardcode this version and parameterize the uwsgi version for the installation with pip.

Besides that, NATS contains two services in its compose:

version: '2.3'

services:
  nats:
    image: docker.elastic.co/integrations-ci/beats-nats:${NATS_VERSION:-2.0.4}-1
    build:
      context: ./_meta
      dockerfile: Dockerfile.2.0.X
      args:
        NATS_VERSION: ${NATS_VERSION:-2.0.4}
    ports:
      - 8222

  nats_1_3:
    image: docker.elastic.co/integrations-ci/beats-nats:${NATS_VERSION:-1.3.0}-1
    build:
      context:./_meta
      dockerfile: Dockerfile.1.3
      args:
        NATS_VERSION: ${NATS_VERSION:-1.3.0}
    ports:
      - 8222

if we pass NATS_VERSION, then both services will be using it, even though the Dockerfile is using different paths and binaries as entrypoint. Is this intended or a bug?

This looks wrong, yes, we should revisit this case.

Also uwsgi contains two services, one for TCP and the other for HTTP traffic. What will happen if we add supported_versions in them? The build/push process on CI will override the first one with the second.

In this case this is in principle ok, we expect both services to run the same version of uwsgi, and we have different tests for each case. We could maybe refactor it as something similar to mysql, with protocol instead of variant.

[jsoriano]

Created elastic/beats#16316 and elastic/beats#16317 to improve UWSGI and NATS docker composes files.

[jsoriano]

Fixes for the dropwizard image in elastic/beats#16332

Thanks @mdelapenya for catching these issues!