Implement pytest fixtures

[waynr]

As a developer of python microservices I often find myself implementing pytest fixtures that yield container objects a la https://github.com/docker/docker-py . Often times what I would really prefer to do is access information about services/containers from within tests to provide access to my service running as a container that has access to other containers in the same network.

The pytest-docker library does this to some extent but it's current implementation and distance from docker-compose makes me think that it may not provide a stable long term interface for doing this. In avast/pytest-docker#20 I have suggested to that author to consider porting his code to this repository, where I think there is a better opportunity to avoid breaking changes as new versions of docker-compose are released in the future by writing tests that run here to ensure such such api changes necessitate changes in the docker pytest fixtures.

[waynr]

So to be clear, I am posting this issue not so much as a suggestion that someone else implement it but to see how open project maintainers would be to a contribution of this nature.

[waynr]

Okay so after peeking around at the docker-compose codebase a little I'm starting to think that it may not be necessary to provide pytest fixtures. Instead it looks like I can make use of compose.config.config.load(...) and compose.project.Project to achieve my end goal which is to make use of a set of services defined in a compose yaml file and manage their up/down lifecycles using pytest fixture scopes. Does that sound like a reasonable approach that's likely not to be borked by api changes?

[shin-]

Hi @waynr,

Our (unpopular) stance on the matter is that we do not make any guarantees about the stability of the Compose API, as the only usecase we officially support is through the CLI. For the same reason, I would have to refuse a contribution of that nature to the repository that would suggest implicit support / guarantees.

Thank you for your understanding!

[waynr]

Thanks for your prompt response @shin-. It looks to me like the API is already tested by virtue of its use in various test cases, but it's unfortunate that the project won't consider supporting a for realz documented and versioned API. I do understand that this would signfiicantly complicate your teams' efforts when it comes to supporting the main product -- the command line tool. But can I be blamed yearning for programmatic access to such a great tool?? ;)

Just out of curiosity is this a hard forever "no"? Are there any circumstances under which your team could be convinced to provide a stable Compose API? Like hypothetically let's say it were easy to do public/protected/private member access in Python, would that make it more likely? I ask because I have this silly little project that I want to eventually flesh out and make production-ready. With a tool like this my goal would be to allow projects like docker-compose to create default access control rules that would more easily limit public visibility of what are internal implementation details while explicitly making others publicly accessible.

For what it's worth I'll probably be satisfied pinning to specific versions of docker-compose. But I do want to drive the point home by showing a test fixture I am working on:

@pytest.yield_fixture(scope="session")
def compose():

    working_dir = functools.reduce(lambda x, _: os.path.dirname(x),
                                   range(4),
                                   __file__)
    cf = config.ConfigFile.from_filename()
    config_details = config.ConfigDetails(working_dir, [cf])
    config_data = config.load(config_details)

    p = project.Project.from_config(config_data)
    p.up()

    yield p

    p.down()


@pytest.fixture(scope="function")
def rabbitmq(compose):
    return compose.get_service("rabbitmq")

I haven't got around to trying it out yet as I have one more fixture to write before I can start writing tests so please forgive if anything looks too funky there; the advantages here are probably obvious but:

• I can exercise direct control over the up/down status of individual services without making calls to the CLI
• I can scale services up or down if I want
• I have access to information about the services and containers using only the names given in the compose yaml file
• probably others i haven't thought of yet, like programmatically modifying volume mounts to point to different "fixture" subdirectories to generate new test cases

Anyway, thanks again for your prompt and considerate response!

[shin-]

Just out of curiosity is this a hard forever "no"? Are there any circumstances under which your team could be convinced to provide a stable Compose API? Like hypothetically let's say it were easy to do public/protected/private member access in Python, would that make it more likely?

Reasons aren't really tech-related, they're a matter of resources and focus. I'm currently the only active maintainer on the project and the effort to define a public API and support the use-case in the long term would be overwhelming and just isn't worth the marginal value it would bring overall. Hope that makes sense!

[waynr]

That's what I figured...thanks for taking the time to respond!