Use external package builder with --installpkg

[sfc-gh-mkeller]

Hey,

When using tox==4.0.0b2 with one of our projects that has an external package env it appears as we cannot supply a wheel file through --installpkg if we have already built one.

$ tox -rvv -e py39 --installpkg dist/snowflake_sqlalchemy-1.3.4-py2.py3-none-any.whl
ROOT: 330 E HandledError| .pkg_external is already defined as a virtualenv-cmd-builder, cannot be virtualenv-pep-517 too [tox/run.py:21]

You can repro the issue with the following commands:

cd /tmp
git clone https://github.com/snowflakedb/snowflake-sqlalchemy
cd snowflake-sqlalchemy
virtualenv -p python3.9 venv
. venv/bin/activate
pip install build
pip install --pre tox
python -m build -w -o dist
tox -rvv -e py39 --installpkg dist/snowflake_sqlalchemy-1.3.4-py2.py3-none-any.whl

Thanks for the help!

[sfc-gh-aling]

hey, I work with Mark on the snowflake-sqlalchemy project and love to share more details as I was debugging this issue:

• the tox will loops the envs list defined in tox.ini and build pkg envs
• when it encounters the env py39 , it found installpkg option is set and will then build a VirtualEnvCmdBuilder for the env .pkg_external
• when it encounters the env fix_lint, because we set skip_install to true in the fix_lint, the tox build a Pep517VirtualEnvPackager for the env .pkg_external
• whichever env (py39 or fix_lint) is built first, the second one would conflict the first one on existing .pkg_external env as defined in the code:
  https://github.com/tox-dev/tox/blob/4.0.0b2/src/tox/session/env_select.py#L264-L267

I'm not very familiar with how .pkg_external module is designed, but just looking at the issue, my gut feeling is that it might be better if each env has its own .pkg_external env or ignore the following attempts to build .pkg_external is there's already one.

[sgbaird]

Installing tox via GitHub actions suddenly stopped working (no changes between commits of the repo: first scheduled test ran fine, next scheduled test resulted in:

ROOT: HandledError| .pkg_external is already defined as a virtualenv-cmd-builder, cannot be virtualenv-pep-517 too
Error: Process completed with exit code 254.

pyscaffold/pyscaffold#683

[gaborbernat]

@sgbaird PR to fix this is welcome 👍

[sgbaird]

@gaborbernat I'll see if I can figure out where the discrepancy is. Any suggestions on pinning the tox version so I can verify a release that is working?

I get the following warning https://github.com/sparks-baird/self-driving-lab-demo/actions/runs/3667147647/jobs/6199430609 which @ssbarnea also mentioned #2409 (comment):

Run pipx run tox --installpkg 'dist/self_driving_lab_demo-0.6.0.post1.dev87+g694e3a0-py3-none-any.whl' -- -rFEx --durations 10 --color yes
creating virtual environment...
installing tox...
ROOT: will run in automatically provisioned tox, host /opt/pipx/.cache/5d372be6c729ea1/bin/python is missing [requires (has)]: tox<4.0.0 (4.0.5)

#2414 is also mentioned, though obviously the warning is still cropping up.

[gaborbernat]

This is likely a version 4 bug, any version 3 should work. Note version 4 is a ground up rewrite so you can't really compare the difference between the two versions.

[sgbaird]

@gaborbernat ah gotcha, I was thinking it had worked in v4 since I had a successful run on 2022-12-07, but I'm realizing what probably happened is the first version installed a v3 (I'm guessing the rc versions, e.g. v4.0.0rc4 won't get installed unless specifically requested) and the next version installed one of the other 2022-12-07 tox releases. I've just been having trouble pinning the version using pipx e.g. https://github.com/sparks-baird/tox-debug/actions/runs/3667384801/jobs/6199817885, which is more specific to PyScaffold's setup.

EDIT: found a temporary workaround using pipx run --spec tox==3.27.1 tox pyscaffold/pyscaffold#683 (comment)

[webknjaz]

I've also encountered this issue. I noticed in one of the backlinked PRs @freakboy3742 suggests that a workaround could be package_env = none — https://github.com/beeware/briefcase/pull/995/files#r1042849461.
Though, I'm not sure what the side effects of this would be, so I'm just linking it for visibility.

[webknjaz]

In that issue, @freakboy3742 claims that the incompatibility is related to using the extras setting in tox.ini but in my case, I don't have it and I still face this problem. I don't have a .pkg_external env defined, either.

Log: https://github.com/ansible-community/ansible-pygments/actions/runs/3679582583/jobs/6224216101#step:11:23.

[freakboy3742]

FWIW: my investigation didn't suggest it was exclusively about extras; it's about having multiple targets in a toxfile, with 2 different install schemes. I'm using --install_pkg on one target, and skip_install on another. This seems to cause Tox confusion because the target being installed from a wheel uses a PEP517 environment, and the skip_install tries to use an old-style install environment (even though, by definition, nothing is being installed); the inconsistency between the two installation approaches is what raises the error.

Adding package_env = with any value on all skip_install targets is enough to avoid the problem. This seems to be enough to convince tox that there's no incompatibility between the skip_install and installed environments because the skip_install targets will use an isolated install environment. The dummy package_env is never created - because nothing is installed - but the validation step passes.

[webknjaz]

@freakboy3742 thanks for the clarification!

👉 CURRENT WORKAROUND FOR PEOPLE COMING FROM GOOGLE 👈

I have verified that setting package_env = ❌ DUMMY NON-EXISTENT ENV NAME ❌ in every toxenv that has skip_install = true works (yes, I tried it out with emojis 🙉, so it raises a red flag if it ever starts to be used by tox).

[takluyver]

If you have CI running on Windows, you may need to get rid of the emoji in the above workaround.

On Azure pipelines, all my jobs started failing, with a warning message saying:

ROOT: No tox.ini or setup.cfg or pyproject.toml found, assuming empty tox.ini at D:\a\1\s

This is because file I/O has the unfortunate default of using a platform-dependent encoding, so it doesn't read files as UTF-8 by default on Windows. Then, because unicode errors subclass from ValueError, the exception is swallowed by this code which is meant to catch missing files.

It took me a while to realise this, even though I'm quite familiar with encoding errors, so I hope this saves someone else some time.

[webknjaz]

If you have CI running on Windows, you may need to get rid of the emoji in the above workaround.

I was facing tracebacks in my case :) Ended up using sed -i to "rename" it under Python 3.6 and Windows. For the newer Pythons, I was able to solve this with the env vars: sphinx-contrib/sphinxcontrib-towncrier@b84a3fa.

[q0w]

Does it really need to check twice if it has external_pkg. In self._register_package_conf() it already sets package=external if --install-pkg is used. This patch seems fix the problem above

--- a/src/tox/tox_env/runner.py
+++ b/src/tox/tox_env/runner.py
@@ -87,9 +87,8 @@ class RunToxEnv(ToxEnv, ABC):
         self._call_pkg_envs("interrupt")
 
     def get_package_env_types(self) -> tuple[str, str] | None:
-        has_external_pkg = getattr(self.options, "install_pkg", None) is not None
-        if self._register_package_conf() or has_external_pkg:
-            has_external_pkg = has_external_pkg or self.conf["package"] == "external"
+        if self._register_package_conf():
+            has_external_pkg = self.conf["package"] == "external"
             self.core.add_config(
                 keys=["package_env", "isolated_build_env"],
                 of_type=str,

[gaborbernat]

I don't think 🤔 it does to check twice.