Ignore extras

[thedrow]

How do I make nox-poetry ignore the extras I have when I session.install(".")?

[cjolowicz]

Hi @thedrow nox-poetry should only install extras that you specify explicitly, using session.install(".[myextra]"). If that's not what happens for you, then it's a bug in nox-poetry and I'd love to have some more detail 😉

[thedrow]

nox-poetry installs all extras:
See https://github.com/cjolowicz/nox-poetry/blob/master/src/nox_poetry/poetry.py#L38 and https://github.com/cjolowicz/nox-poetry/blob/master/src/nox_poetry/poetry.py#L76

I've managed to customize the code. See celery/jumpstarter@04fa573 for a solution.

Unfortunately (and that's another problem), poetry still exports the extra dependency to the requirements file even with my workaround.

[cjolowicz]

nox-poetry activates all extras when exporting the requirements, but it does not install them. The requirements file is used as a constraints file; in other words, it specifies the versions at which packages should be installed, but does not cause any package to be installed per se.

Thanks for the link to your code though. Looking at the pyproject.toml on your branch, I can see that your extra maps to a git dependency. Unfortunately, git dependencies are not supported in constraints files with recent pip. See #141 for a similar issue.

Can you post the error message that you're seeing?

[thedrow]

Same as before as the git dependency is exported.
I have this extra which will turn into a regular dependency once pytransitions/transitions#512 and pytransitions/transitions#515 will be resolved.

I think poetry shouldn't export optional dependencies unless we explictly asked for them.

https://github.com/celery/jumpstarter/pull/23/checks?check_run_id=1878884946#step:6:17

[cjolowicz]

FTR

nox > Command python -m pip install --constraint=.nox/test-3-8/tmp/requirements.txt file:///home/runner/work/jumpstarter/jumpstarter/dist/jumpstarter-0.1.0-py3-none-any.whl failed with exit code 1:
14
DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at pypa/pip#8210.
15
ERROR: Links are not allowed as constraints
16
nox > Session test-3.8 failed.

[thedrow]

Right now I'm bypassing it like this:
celery/jumpstarter@5b690eb

There should be an option to ignore the constraints file.

[cjolowicz]

Right now I'm bypassing it like this:
celery/jumpstarter@5b690eb

session.poetry.session.install is undocumented and might change without notice.

You can use a public API instead:

def install_transitions_from_git(session):
    session.run(
        "python", "-m", "pip", "install", "git+https://git@github.com/pytransitions/transitions.git"
    )

There should be an option to ignore the constraints file.

Hm 🤔 The use case of invoking the original session.install in a session decorated by nox-poetry has come up before (#268). But so far I couldn't think of a good API for that. Do you find using session.run as shown above an acceptable solution for this problem?

BTW I noticed in your noxfile that you're still invoking poetry install in your sessions, after session.install. I suppose you're doing this to get development dependencies installed. I'm wondering though why you need nox-poetry then? You don't seem to want isolated environments containing only, say, documentation builders in the docs session, or test runners in the testing session. And the pinning is already taken care of by poetry install. Not sure by the way what happens to your installed wheel when you perform an editable install with Poetry on top of it. I'm wondering whether your tests still run against the installed package or against the source tree.

[thedrow]

There should be an option to ignore the constraints file.

Hm 🤔 The use case of invoking the original session.install in a session decorated by nox-poetry has come up before (#268). But so far I couldn't think of a good API for that. Do you find using session.run as shown above an acceptable solution for this problem?

No. You need to provide a flag that will bypass nox-poetry so that the API would remain the same.

BTW I noticed in your noxfile that you're still invoking poetry install in your sessions, after session.install. I suppose you're doing this to get development dependencies installed. I'm wondering though why you need nox-poetry then? You don't seem to want isolated environments containing only, say, documentation builders in the docs session, or test runners in the testing session. And the pinning is already taken care of by poetry install. Not sure by the way what happens to your installed wheel when you perform an editable install with Poetry on top of it. I'm wondering whether your tests still run against the installed package or against the source tree.

It's temporary until we fix python-poetry/poetry#1644 and then we can just introduce it to nox-poetry.
Specifying the dependencies twice may make them go out of sync.

python-poetry/poetry#2917 will also help in the next release since I'd be able to only install dev dependencies (and I think nox-poetry should support that as well).

[thedrow]

I think poetry shouldn't export optional dependencies unless we explictly asked for them.

Follow https://github.com/python-poetry/poetry/issues/3451#issuecomment-778090264 for updates.

[cjolowicz]

Thank you for your input!

You need to provide a flag that will bypass nox-poetry so that the API would remain the same.

Let me first make sure I understand what you are proposing.

Option 1: Provide a flag to session.install like this:

@session
def tests(session):
    session.install(
        "git+https://git@github.com/pytransitions/transitions.git",
        constraints=False,
    )

Option 2 (could be combined): Provide a flag to the @session decorator.

@session(constraints=False)
def tests(session):
    session.install(...)         # without constraints
    session.poetry.install(...)  # with constraints

The decorator flag would need to be forwarded through the Session proxy and
ultimately determine the default behavior for session.install.

To be frank, I'm not a big fan of boolean flags. They have their uses, but I
find that often there is a more expressive design waiting around the corner, one
that keeps the interface smaller and that results in less coupling. In fact, I
think I can spot one here.

The other issue I have with this approach is that the interface now diverges
from that provided by Nox. For example, if Nox ever introduces its own
constraints keyword, this package will break. (By the way, this is already the
case with our session.poetry attribute, and I don't like it for that reason.)

[thedrow]

The issues I mentioned prevent me from both using nox and poetry effectively and I hope they'll be resolved soon.

[cjolowicz]

@thedrow Can you please provide clear information on what issue you're having with nox-poetry, as opposed to nox and poetry, as well as a minimal reproducible example? As I said before, nox-poetry does not install extras unless you ask it to, so without further input from your side, I'm going to have to close this. As for the git dependency, your branch does not seem to use it any longer.

I'll also repeat that you're getting minimal benefit from using nox-poetry, seen that you're invoking poetry install in your sessions. Why not just use nox and Poetry on their own?

[cjolowicz]

Closing this as nox-poetry works as designed with respect to extras. I'm happy to reopen this if you provide additional input.

As for the git dependency, let's move this discussion to #141.

Feel free to open a separate feature request if you have an idea for a better API to install packages without constraints.