Kedro build-reqs breaks with pip>=22.0

[luizvbo]

Description

If I run kedro build-reqs with pip>=22.0, an error is raised by piptools. I assume this bug was introduced by the release of the version 22.x of pip on 30/01/2022.

Context

I update the version of my pip package to 22.0.3 and tried to build the requirements of my kedro package with kedro build-reqs, which raises the following error:

/tmp/test/kedro-example/example-venv/bin/python3 -m piptools compile -q /tmp/kedro-example/spaceflights-project/src/requirements.in
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/piptools/__main__.py", line 17, in <module>
    cli()
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/piptools/scripts/compile.py", line 342, in cli
    repository = PyPIRepository(pip_args, cache_dir=cache_dir)
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 106, in __init__
    self._setup_logging()
  File "/tmp/test/kedro-example/example-venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 455, in _setup_logging
    assert isinstance(handler, logging.StreamHandler)
AssertionError

Notice that, when I downgrade pip to 21.3.1, the problem is fixed.

Your Environment

• kedro 0.17.6
• pip 21.3.1
• python 3.8.10

[datajoely]

Hi @luizvbo thanks for the bug report - this is a known issue we have a 0.17.7 patch release coming out shortly :)

[datajoely]

As you say it's also an issue caused downstream by pip-tools jazzband/pip-tools#1558

[antonymilne]

Thank you @luizvbo for the report. As above, this is indeed an issue with pip-tools and pip>=22.

Unfortunately the solution for kedro is not straightforward, because the pip-tools fix that has just been merged (that will go into 6.5.0) also drops support for Python 3.6, which we still support on kedro 0.17.x. In theory we could bump our requirements from pip-tools~=6.4 to pip-tools~=6.5 for Python > 3.6 and pip-tools~=6.4.0 for Python 3.6, but this relies on pip-tools actually releasing 6.5.0 before we release 0.17.7.

I also don't think it's good for us to constrain pip in the kedro requirements, so overall I think we should just leave our requirements how they are with pip-tools~=6.4. Then:

• when pip-tools 6.5.0 is released, so long as you're not using Python 3.6, our requirements of pip-tools~=6.4 is already compatible with this and you can use whichever version of pip you like
• before then, or if you're using Python 3.6, you'll need to use pip-tools==6.4.0 and pip<22

[antonymilne]

Looks like pip-tools 6.5.0 has just been released - I wasn't expecting that to happen so soon! In this case we should be able to bump to pip-tools~=6.5 for Python > 3.6 and pip-tools~=6.4.0 for Python 3.6 in Kedro 0.17.7 🎉