Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-E ignored when using poetry.group #4897

Closed
Conchylicultor opened this issue Dec 15, 2021 · 4 comments
Closed

-E ignored when using poetry.group #4897

Conchylicultor opened this issue Dec 15, 2021 · 4 comments
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@Conchylicultor
Copy link

Conchylicultor commented Dec 15, 2021

Reposting #4891 with more info, as I don't think the issue has been solved.

  • OS version and name: Debian
  • Poetry version: 1.2.0a2
[tool.poetry]
name = "tmp-poetry"
version = "0.1.0"
description = ""
authors = ["xyz <xyz@gmail.com>"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

# Bellow is the bug

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.group.group0]
optional = true

[tool.poetry.group.group0.dependencies]
numpy = "*"
matplotlib = "*"

[tool.poetry.extras]
extra_deps = ["numpy", "matplotlib"]

Issue

When [tool.poetry.extras] is used to refer to deps defined inside [tool.poetry.group.group0.dependencies], those deps are ignored.

Reproduction instrutions:

poetry install -E extra_deps

I would expect the command to install tmp-poetry[extra_deps] which should install numpy. However, this silently fail to install the package.
The extra deps is ignored and not installed, poetry show will display nothing.
poetry run python -c "import numpy" will fail with ModuleNotFoundError

Note: removing the groups and using instead numpy={version="*", optional=true} will work.

Why using dependency group with extra ?

The reason I'm trying to use dependency group is that it greatly reduce boilerplate of defining deps

Without dependency group:

[tool.poetry.dependencies]
absl-py = {version="*", optional=true}
importlib_resources = {version="*", optional=true}
jax = {version="*", optional=true}
jupyter = {version="*", optional=true}
matplotlib = {version="*", optional=true}
numpy = {version=">=1.17", optional=true}
tf-nightly = {version="*", optional=true}
tqdm = {version="*", optional=true}
tree = {version="*", optional=true}
zipp = {version="*", optional=true}

With dependency group:

[tool.poetry.group.group0]
optional = true

[tool.poetry.group.group0.dependencies]
absl-py = "*"
importlib_resources = "*"
jax = "*"
jupyter = "*"
matplotlib = "*"
numpy = "*"
tf-nightly = "*"
tqdm = "*"
tree = "*"
zipp ="*"

Resolving this issue

There are 2 ways to resolve this issue:

  1. Either extras and group are incompatible features, in which case poetry should raise an explicit error message (rather than silently skipping deps installation)
  2. Either extras can use deps defined in group (best), in which case the deps should be correctly installed.

Of course 2) should be greatly preferred.

@Conchylicultor Conchylicultor added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Dec 15, 2021
@Conchylicultor
Copy link
Author

To answer the previous issue objection:

At no point in the doc is explained that extras and group are mutually exclusive: https://python-poetry.org/docs/master/managing-dependencies/#dependency-groups

Poetry provides a way to organize your dependencies by groups. For instance, you might have dependencies that are only needed to test your project or to build the documentation.
Think of dependency groups as labels associated with your dependencies: they don’t have any bearings on whether their dependencies will be resolved and installed by default, they are simply a way to organize the dependencies logically.

The doc say those deps can be used for tests but not that they have to.

Additionally, even global dependencies have a "default" group, which indicates group are not limited to developpement. From the doc:

The dependencies declared in tool.poetry.dependencies are part of an implicit default group.

Additionally, other users seems to understand like me that group and extras features are complementary features which can be used jointly:
#4842

weiji14 added a commit to weiji14/zen3geo that referenced this issue Jun 7, 2022
Turns out groups and extras can't be used together in pyproject.toml by poetry, see python-poetry/poetry#4897. Changed things around so that `pip install .[docs]` works!
weiji14 added a commit to weiji14/zen3geo that referenced this issue Jun 7, 2022
Getting our documentation built on Readthedocs following https://jupyterbook.org/en/stable/publish/readthedocs.html. Also documenting how to build the docs locally.

* 🔧 Configure docs extras dependencies correctly

Turns out groups and extras can't be used together in pyproject.toml by poetry, see python-poetry/poetry#4897. Changed things around so that `pip install .[docs]` works!

* 🔧 Configure readthedocs build

Getting our documentation on Readthedocs! Following https://jupyterbook.org/en/stable/publish/readthedocs.html

* 🔥 Remove deploy-docs GitHub Actions workflow

Just use readthedocs. Sadly this means we can't skip docs build for files outside the docs/ folder anymore.

* 📝 Document how to run things locally

Sneaking in some extra notes on how people can clone the repo, setup a virtual environment, install dependencies and build the jupyter book docs!
@tasansal
Copy link

tasansal commented Jul 8, 2022

I am not sure if this is the right issue to ask this, but here it goes.

I am a little confused about this. I am on the 1.2.0b2 and using dependency groups.

My use case is the following:
1- have minimum dependencies in the default section.
2- have .group.dev.dependencies for dev.
3- have group.my_group.dependencies as optional for installing packages using the --with argument

The groups are not seen as extras if I want to pip install my_package[my_group] from PyPI.

Extras were installed as extras when groups didn't exist using pip. I migrated all my extras into groups, but they are not pip installable.

What is the correct procedure here? Thanks!

@finswimmer
Copy link
Member

Hello,

extras are a python concept to bundle multiple dependencies into one group that can be installed to extend the functionality of the library/app.

groups are a Poetry-only things to organize your non-runtime dependencies. This is why you cannot reference a dependencies listed in an explicite group for an extra.

A PR to improve this in the documentation is always welcome 😃

fin swimmer

@finswimmer finswimmer closed this as not planned Won't fix, can't repro, duplicate, stale Oct 22, 2022
Copy link

github-actions bot commented Mar 1, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Development

No branches or pull requests

3 participants