Skip to content

Commit

Permalink
revert to conda but with libmamaba solver
Browse files Browse the repository at this point in the history
  • Loading branch information
casperdcl committed Sep 12, 2023
1 parent d858fda commit 172d9a1
Show file tree
Hide file tree
Showing 5 changed files with 33 additions and 35 deletions.
15 changes: 8 additions & 7 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@
- docker image updates
- introduce `REMOVE_BUILD_FILES` variable. If set to 1 (which is the default),
most build files will be removed in the docker image.
- introduce `RUN_CTESTS` variable. If set to 1 (which is the default),
- introduce `RUN_CTEST` variable. If set to 1 (which is the default),
the CTests will be run while building the image.
- remove obsolete copying of gadgetron.xml
- Installing requirements for SIRF-Exercises uses its environment.yml or requirements.txt depending on settings.
- Build Gadgetron master. Requires new Ubuntu packages: libdcmtk-dev, libpugixml-dev, libgflags-dev,
libssl-dev, libcurl4-openssl-dev, pkg-config, golang, libboost-coroutine-dev, libboost-context-dev, libboost-random-dev.
- revert to `conda` but with `libmamba` solver
- Build Gadgetron master. Requires new Ubuntu packages: libdcmtk-dev, libpugixml-dev, libgflags-dev,
libssl-dev, libcurl4-openssl-dev, pkg-config, golang, libboost-coroutine-dev, libboost-context-dev, libboost-random-dev.
- Added SuperBuild project dependencies for Gadgetron: range-v3, RocksDB, Date, mrd-storage-server
- updated versions:
- Gadgetron: 42f11bf14b77b16f1ca5bcfbfa435d5ee8cb22a6 (master)
Expand All @@ -25,7 +26,7 @@
- CIL: 0ba2f8e2935db66ec3dfe8c88e407e8d0eb5609f
- CCPi-Regularisation: v22.0.0
- TomoPhantom: v2.0.0

## v3.4.0
- Removed CIL-ASTRA as it has been merged into CIL code base.
- docker images updates
Expand All @@ -52,7 +53,7 @@
- Boost: 1.78.0

## v3.3.1
- VM:
- VM:
- "update_VM.sh -s" (i.e. "UPDATE.sh -s") no longer runs configure_gnome.sh. If you have a very old VM, run it manually instead.
- Updates to run using docker scripts
- installs custom pip and all python prerequisites with pip
Expand All @@ -64,7 +65,7 @@
- no longer force numpy<=1.20
- CMake:
- FindCython allows hints

## v3.3.0
- known problems:
- VM and jupyterhub scripts need merging various fixes
Expand All @@ -78,7 +79,7 @@
- docker:
- fix problems with CUDA repo keys
- minor fixes to scripts for use elsewhere (including preparation for more recent Ubuntu)
- VM:
- VM:
- set BUILD_CIL=ON
- add CITATION.cff (and remove .zenodo.json)
- added numba as dependency in docker files
Expand Down
12 changes: 6 additions & 6 deletions docker/install-sirf-exercises-dep.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import functools

def conda_install(package):
return f'mamba install -y -c conda-forge {package}'.rstrip()
return f'conda install -y -c conda-forge {package}'.rstrip()

def pip_install(package):
return f'pip install -U {package}'.rstrip()
Expand All @@ -30,18 +30,18 @@ def pip_install(package):
else:
print (f'Not on conda-forge: {line.rstrip()}')
install_by_pip.append(line.rstrip())

install_by_conda = functools.reduce(lambda x, y: x + ' ' + y, install_by_conda, '')


install_by_pip = functools.reduce(lambda x, y: x + ' ' + y, install_by_pip, '')


try:
print ("Installing from conda-forge:", install_by_conda)
subprocess.run(conda_install(install_by_conda), shell=True, check=True)
print ("Installing from PyPI:", install_by_pip)
subprocess.run([sys.executable, "-m", pip_install(install_by_pip)], shell=True, check=True)

except subprocess.CalledProcessError as cpe:
print (cpe)
print (cpe)
1 change: 0 additions & 1 deletion docker/requirements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ channels:
- ccpi
- defaults
dependencies:
- mamba
- setuptools
- wheel
- pytest
Expand Down
24 changes: 11 additions & 13 deletions docker/user_python-ubuntu.sh
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
#!/usr/bin/env bash

# User can pass 2 parameters to this script:
# User can pass 2 parameters to this script:
# 1. PYTHON_EXECUTABLE which can be either the 'miniconda' string or the location of the Python executable
# 2. PYTHON_INSTALL_DIR which points to the location where the user has their virtual environment installed
# The virtual environment can be either virtualenv or conda
#
# To pass these parameters the user needs to set (at least) temporary environment variables:
# PYTHON_EXECUTABLE=python3 PYTHON_INSTALL_DIR="~/virtualenv" bash user_python-ubuntu.sh
# For details see https://github.com/SyneRBI/SIRF-SuperBuild/pull/692#issuecomment-1102704682
# For details see https://github.com/SyneRBI/SIRF-SuperBuild/pull/692#issuecomment-1102704682
[ -f .bashrc ] && . .bashrc
set -exv

if [ -n "${PYTHON_EXECUTABLE}" ]; then
if [ -n "${PYTHON_EXECUTABLE}" ]; then
PYTHON=$PYTHON_EXECUTABLE
else
PYTHON='miniconda'
Expand All @@ -26,16 +26,14 @@ fi
# Python
case "$PYTHON" in
miniconda)
#curl -L https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > miniconda.sh
curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh > miniconda.sh
curl -L https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > miniconda.sh
echo -e "\nyes\n${INSTALL_DIR}\nno" | bash miniconda.sh
rm miniconda.sh
source "$INSTALL_DIR"/bin/activate
#conda config --add channels conda-forge
# https://github.com/conda/conda/issues/6030
#conda update -c conda-forge -y conda
# https://github.com/SyneRBI/SIRF-SuperBuild/issues/826
#conda install -c conda-forge -c defaults -y mamba
conda config --add channels conda-forge
conda update -c conda-forge -y conda
conda install conda-libmamba-solver
conda config --set solver libmamba
;;
*python*)
# virtualenv
Expand All @@ -54,11 +52,11 @@ esac

if [ "$PYTHON" = "miniconda" ]; then
if [ -f requirements.yml ]; then
# installs the required packages in the environment with requirements.yml.
# installs the required packages in the environment with requirements.yml.
# Notice that SciPy is set to 1.7.3 to prevent `GLIBCXX_3.4.30' not found
mamba env update --file requirements.yml
conda env update --file requirements.yml
fi
mamba clean -y --all
conda clean -y --all
# Python (runtime)
else
if [ -f requirements.txt ]; then
Expand Down
16 changes: 8 additions & 8 deletions docker/user_service-ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
[ -f .bashrc ] && . .bashrc
set -ev
INSTALL_DIR="${1:-/opt}"
if [ -n "${PYTHON_EXECUTABLE}" ]; then
if [ -n "${PYTHON_EXECUTABLE}" ]; then
PYTHON=$PYTHON_EXECUTABLE
else
PYTHON='miniconda'
Expand All @@ -13,11 +13,11 @@ git clone https://github.com/SyneRBI/SIRF-Exercises --recursive -b master $INSTA

if [ "$PYTHON" = "miniconda" ]; then
if [ -f requirements-service.yml ]; then
# installs the required packages in the environment with requirements-service.yml.
# installs the required packages in the environment with requirements-service.yml.
# Notice that these requirements TEMPORARILY contains also the packages for SIRF-Exercises
mamba env update --file requirements-service.yml
conda env update --file requirements-service.yml
fi
mamba clean -y --all
conda clean -y --all

# Python (runtime)
else
Expand All @@ -30,13 +30,13 @@ fi
cd $INSTALL_DIR/SIRF-Exercises
if [ "$PYTHON" = "miniconda" ]; then
if [ -f environment.yml ]; then
mamba env update --file environment.yml
conda env update --file environment.yml
else
if [ -f requirements.txt ]; then
cat requirements.txt
# installing the requirements.txt with mamba requires some cleaning of the requirements.txt
# installing the requirements.txt with conda requires some cleaning of the requirements.txt
# Also the requirements.txt contains some packages that are not found on conda-forge, i.e. brainweb
# Therefore, these need to be installed by pip.
# Therefore, these need to be installed by pip.
# This is handled by the install-sirf-exercises-dep.py script
python ~/install-sirf-exercises-dep.py requirements.txt
else
Expand All @@ -58,7 +58,7 @@ git config --global filter.nbstripout.extrakeys '
metadata.language_info.pygments_lexer metadata.language_info.version'

#install nbstripout in the SIRF-Exercises repo
cd $INSTALL_DIR/SIRF-Exercises
cd $INSTALL_DIR/SIRF-Exercises
nbstripout --install
# jupyter labextension install @jupyter-widgets/jupyterlab-manager

Expand Down

0 comments on commit 172d9a1

Please sign in to comment.