Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update snakemake Python dependency #42147

Merged
merged 1 commit into from
Jul 24, 2023
Merged

Update snakemake Python dependency #42147

merged 1 commit into from
Jul 24, 2023

Conversation

wm75
Copy link
Contributor

@wm75 wm75 commented Jul 24, 2023

7.30.2 requires at least Python 3.9

I hope that's correct @johanneskoester?

7.30.2 requires at least Python 3.9
@wm75 wm75 mentioned this pull request Jul 24, 2023
@tsibley
Copy link
Contributor

tsibley commented Jul 24, 2023

I concur with this PR. I noticed the issue when some of our CI with Python 3.8 + Snakemake 7.30.2 from Bioconda started failing with:

Traceback (most recent call last):
  File "…/bin/snakemake", line 6, in <module>
    from snakemake import main
  File "…/lib/python3.8/site-packages/snakemake/__init__.py", line 9, in <module>
    raise ValueError("Snakemake requires at least Python 3.9.")
ValueError: Snakemake requires at least Python 3.9.

when 7.30.2 should never have been selected by the solver for installation on 3.8 to begin with.

@tsibley
Copy link
Contributor

tsibley commented Jul 24, 2023

@BiocondaBot please add label

@BiocondaBot BiocondaBot added the please review & merge set to ask for merge label Jul 24, 2023
@corneliusroemer
Copy link
Member

Verified correctness of this PR, see
snakemake/snakemake#2341 (comment)
snakemake/snakemake#2370
snakemake/snakemake#2369

@corneliusroemer corneliusroemer merged commit 0c4954d into master Jul 24, 2023
5 checks passed
@corneliusroemer corneliusroemer deleted the snakemake-py39 branch July 24, 2023 19:44
@corneliusroemer
Copy link
Member

I don't know enough about build numbers to know whether Python 3.8 will now try to pick build 0 (before the version requirement bump was part of conda metadata).

@tsibley
Copy link
Contributor

tsibley commented Jul 24, 2023

Indeed the solver (well, at least libmamba's solver) will just pick the build 0 instead. :-(

I thought it would, but tested anyway and was initially surprised because snakemake ==7.30.2 hdfd78af_1 (the build from this PR) got installed. Unfortunately it was able to be installed because it pulled in the pyhdfd78af_0 build of the snakemake-minimal package of the same version.

Can one yank releases/builds from Bioconda? I believe Anaconda's repositories let you do this, but who can do that for Bioconda?

Separately, it seems this pattern of the snakemake Conda package depending on the snakemake-minimal "sub" package should not only pin the exact same version as it does now but also the exact same build string. This doesn't functionally change the situation here, but it would likely help avoid other build mismatch issues in the future.

@tsibley
Copy link
Contributor

tsibley commented Jul 24, 2023

Until someone can yank these files from Bioconda's channel:

snakemake-7.30.2-hdfd78af_0.tar.bz2
snakemake-minimal-7.30.2-pyhdfd78af_0.tar.bz2

then anyone with a Conda env with Python 3.7 or 3.8 trying to install Snakemake (without a version restriction) will get a broken version.

A workaround is to exclude 7.30.2, e.g.:

micromamba install 'snakemake !=7.30.2'

but this version restriction will be required in perpetuity (assuming Snakemake doesn't reintroduce support for 3.7 or 3.8).

tsibley added a commit to nextstrain/cli that referenced this pull request Jul 29, 2023
…nt runtime setup

This should fix our CI.

The initial Bioconda builds (build number 0) for Snakemake 7.30.2 missed
a major bump in Snakemake's minimum Python version (3.7 → 3.9).  This
rendered the Bioconda packages for snakemake and snakemake-minimal
installable on 3.7 or 3.8 but non-functional at runtime.  Subsequent
builds (build number 1 and onwards) of 7.30.2 fixed this oversight, but
since the initial builds haven't been yanked/marked broken, the solver
still chooses them.¹

Since Conda's package spec syntax doesn't allow for exclusion of
specific builds, exclude all of 7.30.2.  It's fine if we never test on
this patch version²; Python 3.6–3.8 will get <7.30.2 and 3.9 and onwards
will get the already-available subsequent version 7.31.0.

¹ <bioconda/bioconda-recipes#42147>

² It looks like SemVer, but it ain't!  The major version change for
  Python occurred between 7.30.1 and 7.30.2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please review & merge set to ask for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants