Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use stand-alone absl-py as dependency for jax w/ gfbf/2023a + CUDA 12.1.1 + add Optax v0.2.2, Flax v0.8.4, scvi-tools v1.1.2 #21039

Merged

Conversation

lexming
Copy link
Contributor

@lexming lexming commented Jul 24, 2024

(created using eb --new-pr)

Equivalent to #21038 for stack with CUDA

Adding new easyconfig for Flax, which deserves its own package; and for absl-py, which is used in many places already.

This clarifies the dependency tree of the Flax/JAX stack as:
absl-py > jax > Optax > Flax

Changelog:

  • simplify easyconfig of jax v0.4.25:
    • replace component on absl-py with regular dependency
    • define source_urls individually per each source sownloaded
  • add easyconfig for Optax v0.2.2 with CUDA 12 without those extensions already provided by its dependencies
  • add easyconfig for Flax v0.8.4 with CUDA 12
  • add easyconfig for scvi-tools v1.2.2 with CUDA 12 without those extensions already provided by its dependencies

@lexming
Copy link
Contributor Author

lexming commented Jul 26, 2024

Test report by @lexming
SUCCESS
Build succeeded for 5 out of 5 (5 easyconfigs in total)
node402.hydra.os - Linux Rocky Linux 8.10, x86_64, AMD EPYC 7282 16-Core Processor (zen2), 1 x NVIDIA NVIDIA A100-PCIE-40GB, 550.90.07, Python 3.6.8
See https://gist.github.com/lexming/80f6e230f828d57a0dea58141c4566bd for a full test report.

@boegel
Copy link
Member

boegel commented Aug 29, 2024

@boegelbot please test @ jsc-zen3-a100

@boegelbot
Copy link
Collaborator

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=21039 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_21039 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 4774

Test results coming soon (I hope)...

- notification for comment with ID 2317739963 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 5 out of 5 (5 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.18
See https://gist.github.com/boegelbot/222dfbe77bd170f61ff05b35c991d99b for a full test report.

@boegel boegel changed the title Flax/JAX stack in 2023a w/ CUDA 12.1.1: scvi-tools, Flax, Optax, jax, absl-py use stand-alone absl-py as dependency for jax w/ gfbf/2023a + CUDA 12.1.1 + add Optax v0.2.2, Flax v0.8.4, scvi-tools v1.1.2 Sep 4, 2024
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Member

boegel commented Sep 4, 2024

Going in, thanks @lexming!

@boegel boegel merged commit fa1afbe into easybuilders:develop Sep 4, 2024
9 checks passed
@lexming lexming deleted the 20240724124338_new_pr_absl-py210 branch September 5, 2024 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants