From 595139cee783518cccd15125925012a3209a76f9 Mon Sep 17 00:00:00 2001 From: Stephen Finucane Date: Tue, 8 Dec 2020 10:20:19 +0000 Subject: [PATCH] Fix lower-constraints job pip 20.3 finally includes a proper dependency resolver. Its use is causing the following error messages on the lower-constraints job: ERROR: Cannot install ... because these package versions have conflicting dependencies. The conflict is caused by: bandit 1.1.0 depends on PyYAML>=3.1.0 cliff 3.4.0 depends on PyYAML>=3.12 openstacksdk 0.52.0 depends on PyYAML>=3.13 Bump our lower constraint for PyYAML to resolve this issue. With that resolved, we see a new issue: ERROR: Could not find a version that satisfies the requirement cryptography>=2.7 (from openstacksdk) ERROR: No matching distribution found for cryptography>=2.7 This is less self-explanatory but looking at the lower-constraints for openstacksdk 0.52.0 shows a dependency on cryptography 2.7 [1], meaning we need to bump this also. Next up, flake8-import-order seems to cause the dependency resolver to go nuts, eventually ending with the following error message in a Python 3.6 environment: Using cached enum34-1.1.2.zip (49 kB) ERROR: Command errored out with exit status 1: command: ... cwd: ... Complete output (9 lines): Traceback (most recent call last): File "", line 1, in File ".../lib/python3.6/site-packages/setuptools/__init__.py", line 7, in import setuptools.distutils_patch # noqa: F401 File ".../lib/python3.6/site-packages/setuptools/distutils_patch.py", line 9, in import re File "/usr/lib64/python3.6/re.py", line 142, in class RegexFlag(enum.IntFlag): AttributeError: module 'enum' has no attribute 'IntFlag' ---------------------------------------- A quick Google suggests this is because the enum34 package is not complete [2]. We shouldn't even be using it since our base virtualenv should at least use Python 3.6, but I guess some dependency doesn't properly restrict the dependency to <= Python 3.4. This is moved from 'test-requirements.txt' to 'tox.ini' since we don't need to use our constraints machinery for linters. Finally, the versions of bandit and hacking that pip is bringing in both requires in a newer version of babel, which in turn requires a new version of pytz. Collecting hacking>=2.0.0 ... ERROR: Cannot install oslo.i18n because these package versions have conflicting dependencies. The conflict is caused by: babel 2.9.0 depends on pytz>=2015.7 babel 2.8.1 depends on pytz>=2015.7 babel 2.8.0 depends on pytz>=2015.7 babel 2.7.0 depends on pytz>=2015.7 Seeing as we shouldn't be tracking bandit in lower-constraints, I'm not sure why we're want to bump these dependencies for just that. As above, we move these dependencies out of 'test-requirements' and into 'tox.ini' since we can do that for linters. Conflicts: lower-constraints.txt test-requirements.txt NOTE(stephenfin): Conflicts are due to the absence of ddt and presence of mock in lower-constraints.txt and test-requirements.txt, respectively. Modifications: lower-constraints.txt NOTE(stephenfin): There's no need to bump cryptography here since we're using an older version of openstacksdk. [1] https://opendev.org/openstack/openstacksdk/src/tag/0.52.0/requirements.txt#L19 [2] https://github.com/iterative/dvc/issues/1995#issuecomment-491889669 Change-Id: I8ec738fbcabc8d8553db79a876e5592576cd18fa Signed-off-by: Stephen Finucane (cherry picked from commit 20769cd7b27d51da84a324a17922427eba5c6eac) (cherry picked from commit 83cd9b5b9c3b4c471b41190675f880599b78e44e) --- lower-constraints.txt | 2 +- test-requirements.txt | 3 --- tox.ini | 8 ++++++-- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/lower-constraints.txt b/lower-constraints.txt index 1fa30674e1..927faaea42 100644 --- a/lower-constraints.txt +++ b/lower-constraints.txt @@ -113,7 +113,7 @@ python-watcherclient==2.5.0 python-zaqarclient==1.0.0 python-zunclient==3.6.0 pytz==2013.6 -PyYAML==3.12 +PyYAML==3.13 repoze.lru==0.7 requests-mock==1.2.0 requests==2.14.2 diff --git a/test-requirements.txt b/test-requirements.txt index 55ae1ea459..2e56f4be39 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,10 +1,8 @@ # The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. -hacking>=2.0.0 # Apache-2.0 coverage!=4.4,>=4.0 # Apache-2.0 fixtures>=3.0.0 # Apache-2.0/BSD -flake8-import-order>=0.13 # LGPLv3 mock>=2.0.0 # BSD oslotest>=3.2.0 # Apache-2.0 requests>=2.14.2 # Apache-2.0 @@ -15,5 +13,4 @@ stestr>=1.0.0 # Apache-2.0 testtools>=2.2.0 # MIT tempest>=17.1.0 # Apache-2.0 osprofiler>=1.4.0 # Apache-2.0 -bandit!=1.6.0,>=1.1.0 # Apache-2.0 wrapt>=1.7.0 # BSD License diff --git a/tox.ini b/tox.ini index 163a9a2474..ace3506a04 100644 --- a/tox.ini +++ b/tox.ini @@ -28,9 +28,13 @@ commands = {toxinidir}/tools/fast8.sh [testenv:pep8] +deps = + hacking>=2.0.0,<4.0.0 + bandit!=1.6.0,>=1.1.0 + flake8-import-order>=0.13 # LGPLv3 commands = - flake8 - bandit -r openstackclient -x tests -s B105,B106,B107,B401,B404,B603,B606,B607,B110,B605,B101 + flake8 + bandit -r openstackclient -x tests -s B105,B106,B107,B401,B404,B603,B606,B607,B110,B605,B101 [testenv:bandit] # This command runs the bandit security linter against the openstackclient