Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docker instructions #30

Merged
merged 3 commits into from
Feb 24, 2023
Merged

Conversation

miguelsimon
Copy link
Owner

Several people have reported issues with dependencies in different execution contexts eg. M1 macs, windows.

This adds instructions for running in a Dockerized context on the standard python:3.10 docker image.

Related to #29

@valentinboyanov
Copy link
Collaborator

This is strange, it seems that it couldn't find jaxlib:

ERROR: Could not find a version that satisfies the requirement jaxlib (from versions: none)
ERROR: No matching distribution found for jaxlib

Full log:

➜  supervised_learning git:(supervised-learning-environment) docker run -it --rm \                          
        -v $(pwd):/supervised_learning \
        -p 8888:8888 \
        --workdir /supervised_learning \
        python:3.10 /bin/bash
root@092958694e34:/supervised_learning# python3 -m venv env
env/bin/pip install -r requirements.txt
env/bin/jupyter notebook --no-browser --allow-root --ip=0.0.0.0 --port=8888
Collecting jupyter
  Downloading jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)
Collecting scipy==1.9
  Downloading scipy-1.9.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.1/40.1 MB 1.1 MB/s eta 0:00:00
Collecting pandas
  Downloading pandas-1.5.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.4/11.4 MB 1.3 MB/s eta 0:00:00
Collecting matplotlib
  Downloading matplotlib-3.7.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 1.3 MB/s eta 0:00:00
Collecting numpy
  Downloading numpy-1.24.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 1.3 MB/s eta 0:00:00
Collecting mypy
  Downloading mypy-1.0.1-py3-none-any.whl (2.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 1.3 MB/s eta 0:00:00
Collecting jax
  Downloading jax-0.4.4.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 1.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
ERROR: Ignored the following versions that require a different python version: 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.0rc1 Requires-Python >=3.7,<3.10; 1.7.0rc2 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10
ERROR: Could not find a version that satisfies the requirement jaxlib (from versions: none)
ERROR: No matching distribution found for jaxlib

[notice] A new release of pip available: 22.3.1 -> 23.0.1
[notice] To update, run: python3 -m pip install --upgrade pip
bash: env/bin/jupyter: No such file or directory
root@092958694e34:/supervised_learning# 

I will investigate if it's related to the docker image type.

@miguelsimon
Copy link
Owner Author

hmm, it worked on an intel mac & on a windows machine, things that could be happening:

  • a preexisting env/ directory might be confusing pip: rm -r env before running the docker command to make sure
  • maybe a cpu-related problem because pip can't find binaries for M1?

@valentinboyanov
Copy link
Collaborator

maybe a cpu-related problem because pip can't find binaries for M1?

My gut feeling is telling me this one.

But it shouldn't be an issue because I'm able to install the requirements without docker:

  DEPRECATION: jax is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for jax ... done

@miguelsimon
Copy link
Owner Author

Probably cpu: jax-ml/jax#13608

I can try using conda instead of pip, I'll ping you when that's done.

@miguelsimon
Copy link
Owner Author

I've looked for a simple way to dockerize a universal context but conda requires specifying the CPU arch so it gets messy.

So I've opted for adding a warning in the docs and a suggestion to try conda if you're running on arm-linux.

Copy link
Collaborator

@valentinboyanov valentinboyanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@miguelsimon miguelsimon merged commit fc9ee99 into master Feb 24, 2023
@miguelsimon miguelsimon deleted the supervised-learning-environment branch February 24, 2023 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants