tox in parallel leaks TOXENV to subprocess #1444
Labels
bug:normal
affects many people or has quite an impact
help:wanted
Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Comments
hroncok
changed the title
gaborbernat
changed the title
gaborbernat
added
the
help:wanted
|
We use TOXENV to signal to the sub-process child what environment to run. Agreed that subprocesses from within that parallel run should not pass it along. PR welcomed! |
|
It seem that TOX_PARALLEL_ENV is acually used. And tox respects this even when not run in parallel. An obvious workaround for this problem is to not respect TOX_PARALLEL_ENV when not being a paralllel worker. |
|
tox recognizes itself as a parallel worker if TOX_PARALLEL_ENV is set. |
|
Removing this line from passenv fixes the problem: tox/src/tox/config/__init__.py Line 673 in 1883946 I'm investigating why it is in there in the first place. |
|
This was added in #1139 and fixing it would contradict the entire purpose of that PR. Not sure how to proceed, please advise. |
|
Solution ideas:
|
|
I would suggest proposal 1 🤔 |
|
Any name suggestions? |
|
|
|
_TOX_PARALLEL_ENV is sane enough 👍 the underscore shows its private |
|
Working on it, but will not yet have a PR ready today. |
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 21, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 21, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
6 tasks
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default).
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 22, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 23, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 23, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 23, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
hroncok
added a commit
to hroncok/tox
that referenced
this issue
Oct 23, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: tox-dev#1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes tox-dev#1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
gaborbernat
pushed a commit
that referenced
this issue
Nov 11, 2019
...but keep the TOX_PARALLEL_ENV variable for others to consume. The problem with using environment variables to let tox know it's being run as a parallel worker is the following: Either you don't pass the environment variable, but the test frameworks and test suites are unaware that tox is running in parallel. This makes it hard for cases like pytest-django: #1139 Or you pass the environment variable, but it makes tox invoked within tox think it is a parallel worker even when it isn't. This makes it hard to test various tox plugins that invoke tox in their integration test suite: Fixes #1444 By introducing two variables we can use one ("private", _TOX_PARALLEL_ENV) to control the behavior of tox (and don't pass it) and another ("public", TOX_PARALLEL_ENV) to let the tests and test frameworks know tox is running in parallel (and pass it by default). The integration test is a tad complicated invoking subprocess.Popen instead of subprocess.run, to support Python 2.7 and 3.4.
|
This now has been released under https://tox.readthedocs.io/en/latest/changelog.html#v3-14-1-2019-11-13 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
When running tox from within tox in parallel mode, the tox env of the inner tox is changed unlike when running it without parallel mode. This used to work with detox.
Get the latest tox
Create an inner project with simple tox.ini
See that invoking the inner dir's tox via subprocess works
Create an outer tox.ini
Run tox in parallel
The text was updated successfully, but these errors were encountered: