group envs into an alias

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/238
• Originally reported by: mmerickel
• Originally created at: 2015-04-20T07:51:01.615

We have always used tox to run our coverage commands. However recently we started running coverage on py2 and py3 and combining it into a single coverage report that should be 100%.

Anyway, this is done using 3 environments. One for py2, one for py3 and one to aggregate the results.

This is all fine but the interface has changed from tox -e cover to tox -e py2-cover,py3-cover,cover. Ideally tox would possibly support some sort of grouping instead of requiring us to wrap this invocation in something else.

But wait, we found a scary solution!

[testenv:py2-cover]
commands =
    coverage run ...
setenv =
    COVERAGE_FILE=.coverage.py2

[testenv:py3-cover]
commands =
    coverage run ...
setenv =
    COVERAGE_FILE=.coverage.py3

[textenv:cover]
commands =
    coverage erase
    tox -e py2-cover
    tox -e py3-cover
    coverage combine
    coverage xml
setenv =
    COVERAGE_FILE=.coverage
deps =
    tox

A recursive tox file! Anyway this has some downsides like tox -r is not propagated downward to the sub-toxes. However this solves our issues with the CLI, and even gives us the opportunity to do things prior to the sub-toxes like erase without introducing yet another tox env.

I don't have an actual proposal but I wanted to open an issue and get some thoughts on possible solutions inside or outside of tox.

[pytoxbot]

Original comment by @nedbat

I have a similar need. My idea would be to have "tox -e cover" mean, run all the envs with a "cover" factor in them.

Just as a configuration line like "cover: foo" applies to all envs with a cover factor, "-e cover" means, run all of those envs.

I would find this very helpful. It would let me test all of my py27 runs, or all of my pytracer runs with a simple invocation.

[obestwalter]

Wow. So we have tox, detox, retox (recursive tox) and maybe gtox (grouped tox). I have no opinion about this yet, so just marking it as a potential enhancement for now.

[obestwalter]

Please also see #223 for some further ideas on the topic.

[warsaw]

So one of the things from #223 that's important I think is that there be a "default" group that gets run when you say tox with no other options. The other idea from there was that you could say tox -g grouname to run that group's environments.

[warsaw]

I'm started to really like this idea. I think it nicely solves #418 too (at least, my motivating use case for that issue).

[ssbarnea]

Any update on this? I have a much simpler use case: I just need an alias because we refactored a "pep8" environment into a more generic "linters" and it would be very convenient to be able to give aliases for people that are still in the habbit of doing "tox -e pep8".

Now I am using placeholder environments but this is too verbose, a real alias would be better.

[gaborbernat]

I think blocked mostly on help wanted for now. Aka people available to implement it.

[jugmac00]

Meanwhile... this looks like a viable workaround: https://stackoverflow.com/a/65516865/672833

[humitos]

I was looking for a way to select only a subset of environments to run today as well, and I ended up doing:

1. list all the environments that starts with py36

$ tox -l | grep -e "^py36"
py36-sphinx34
py36-sphinx35
py36-sphinxlatest

2. concatenate them with paste

$ tox -l | grep -e "^py36" | paste -sd ","
py36-sphinx34,py36-sphinx35,py36-sphinxlatest

3. pass this list to tox

$ tox -e $(tox -l | grep -e "^py36" | paste -sd ",")
...

I'm not proud of this, tho 😄

[ssbarnea]

Using stdout output from tox is risky as it can easily spill some other undesired stuff, like when it does bootstrap itself when plugins are required.

[gaborbernat]

For anyone wondering I'm still working on this, but this requires a major rework of tox 4 how it registers and handles environments, so might take a few more weeks, but WIP...

[gaborbernat]

resolved by #2290