CI: add docker images (#1633)

.github/workflows/README.md

@@ -57,3 +57,15 @@ To get the action to work I first had to:
 2. [Set the source](https://github.com/TomographicImaging/CIL/settings/pages) for our github pages to be the gh-pages branch.
 
 I followed the examples on the [sphinx build action page](https://github.com/marketplace/actions/sphinx-build), specifically this [example workflow](https://github.com/ammaraskar/sphinx-action-test/blob/master/.github/workflows/default.yml)
+
+## Building/Pushing the Docker Image: docker job
+
+This builds a docker image using the `Dockerfile` at the root of this repository.
+
+The image is also pushed to `ghcr.io/tomographicimaging/cil:TAG` (https://github.com/TomographicImaging/CIL/pkgs/container/cil), where `TAG` is given by:
+
+git ref | docker tag(s)
+--|--
+`master` branch | `master`
+`vM.m.p` tag | `M.m.p`, `M.m`, `latest`
+anything else | not pushed (only built)

.github/workflows/build.yml

@@ -95,3 +95,39 @@ jobs:
         branch: gh-pages
         dir: docs/build
         nojekyll: true
+  docker:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        fetch-depth: 0
+        ref: ${{ github.event.pull_request.head.sha || github.ref }} # fix SHA
+        submodules: recursive
+    - uses: docker/setup-buildx-action@v3
+    - uses: docker/metadata-action@v5
+      id: meta
+      with:
+        images: ghcr.io/${{ github.repository }}
+        tags: |
+          type=ref,event=branch
+          type=semver,pattern={{version}}
+          type=semver,pattern={{major}}.{{minor}}
+        labels: |
+          org.opencontainers.image.licenses=Apache-2.0 AND BSD-3-Clause AND GPL-3.0
+    - uses: docker/login-action@v3
+      if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags')
+      with:
+        registry: ghcr.io
+        username: ${{ github.actor }}
+        password: ${{ secrets.GITHUB_TOKEN }}
+    - uses: docker/build-push-action@v5
+      with:
+        cache-from: type=gha
+        cache-to: type=gha,mode=max
+        context: .
+        push: ${{ github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags') }}
+        tags: ${{ steps.meta.outputs.tags }}
+        labels: ${{ steps.meta.outputs.labels }}

CHANGELOG.md

@@ -1,22 +1,22 @@
 * x.x.x
-  - Added a weight argument to the L1Norm function 
+  - Added a weight argument to the L1Norm function
   - Allow reduction methods on the DataContainer class to accept axis argument as string which matches values in dimension_labels
-  - Added the functions `set_norms` and `get_norms` to the `BlockOperator` class 
+  - Added the functions `set_norms` and `get_norms` to the `BlockOperator` class
   - Internal variable name change in BlockOperator to aid understanding
-  - Fixed formatting errors in the L2NormSquared and LeastSquares documentation 
+  - Fixed formatting errors in the L2NormSquared and LeastSquares documentation
   - Bug fix for BlockDataContainer as iterator
   - Dropped support for IPP versions older than 2021.10 due to header changes
   - Fix build include directories
   - Proximal of MixedL21Norm with numpy backend now accepts numpy ndarray, DataContainer and float as tau parameter
   - ZeroOperator no longer relies on the default of allocate
   - Bug fix in SIRF TotalVariation unit tests with warm_start
   - Allow show2D to be used with 3D `DataContainer` instances
-  - Update documentation for symmetrised gradient 
+  - Update documentation for symmetrised gradient
   - Added documentation for CompositionOperator and SumOperator
   - Updated FISTA and ISTA algorithms to allow input functions to be None
   - Bug fix in the adjoint of the Diagonal Operator for complex values
-
-
+  - Update conda build action to v2 for 2.5x quicker builds
+  - Add docker image & push to [`ghcr.io/tomographicimaging/cil`](https://github.com/TomographicImaging/CIL/pkgs/container/cil)
 
 * 23.1.0
   - Fix bug in IndicatorBox proximal_conjugate
@@ -120,11 +120,11 @@
     - Update to version 0.1.1 of lauramurgatroyd/build-sphinx-action for building the documentation - ensures docs are always built from cil master
 
 * 21.4.1
- - Removed prints from unittests and cleanup of unittest code.
- - CMake:
-   - install script re-allows selection of non default install directory ([#1246](https://github.com/TomographicImaging/CIL/issues/1246))
- - TIFF writer uses logging
- - Added unittests for TIFF functionality
+  - Removed prints from unittests and cleanup of unittest code.
+  - CMake:
+    - install script re-allows selection of non default install directory ([#1246](https://github.com/TomographicImaging/CIL/issues/1246))
+  - TIFF writer uses logging
+  - Added unittests for TIFF functionality
 
 * 21.4.0
   - PEP 440 compliant version
@@ -231,7 +231,7 @@
   - introduction of Algorithm class
   - unit test expanded and moved to test directory
   - unified build system on Jenkins based on CCPi-VirtualMachine repo
- - switched to calendar versioning YY.0M.
+  - switched to calendar versioning YY.0M.
 
 * 0.10.0
 

Dockerfile

@@ -0,0 +1,39 @@
+# sources:
+# - https://github.com/jupyter/docker-stacks
+# - https://github.com/TomographicImaging/CIL#installation-of-cil
+# consumers:
+# - https://github.com/stfc/cloud-docker-images
+# TODO: use `ubuntu-22.04` after `python=3.11` is supported, vis.
+# https://github.com/TomographicImaging/CIL/issues/1490
+FROM jupyter/tensorflow-notebook:ubuntu-20.04
+LABEL org.opencontainers.image.source=https://github.com/TomographicImaging/CIL
+# tigre: BSD-3-Clause, astra-toolbox: GPL-3.0
+LABEL org.opencontainers.image.licenses="Apache-2.0 AND BSD-3-Clause AND GPL-3.0"
+
+# CUDA-specific packages
+ARG CIL_EXTRA_PACKAGES=tigre astra-toolbox
+# build & runtime dependencies
+# TODO: sync scripts/create_local_env_for_cil_development.sh, scripts/requirements-test.yml, recipe/meta.yaml (e.g. missing libstdcxx-ng _openmp_mutex pip)?
+# vis. https://github.com/TomographicImaging/CIL/pull/1590
+COPY --chown="${NB_USER}" scripts/requirements-test.yml environment.yml
+# channel_priority: https://stackoverflow.com/q/58555389
+RUN sed -ri '/tigre|astra-toolbox/d' environment.yml \
+  && for pkg in jupyter-server-proxy $CIL_EXTRA_PACKAGES; do echo "  - $pkg" >> environment.yml; done \
+  && conda config --env --set channel_priority strict \
+  && for ch in defaults ccpi intel conda-forge; do conda config --env --add channels $ch; done \
+  && mamba env update -n base \
+  && mamba clean -a -y -f \
+  && rm environment.yml \
+  && fix-permissions "${CONDA_DIR}" \
+  && fix-permissions "/home/${NB_USER}"
+
+# NB: trailing `/` is required
+ENV TENSORBOARD_PROXY_URL=/user-redirect/proxy/6006/
+
+# build & install CIL
+COPY --chown="${NB_USER}" . src
+RUN mkdir build && cd build \
+  && cmake ../src -DCMAKE_BUILD_TYPE="Release" -DCONDA_BUILD=ON -DCMAKE_INSTALL_PREFIX="${CONDA_DIR}" \
+  && cmake --build . --target install \
+  && cd .. && rm -rf src build \
+  && fix-permissions "${CONDA_DIR}"

README.md

@@ -47,6 +47,17 @@ CIL's [optimised FDK/FBP](https://github.com/TomographicImaging/CIL/discussions/
 1. the Intel [Integrated Performance Primitives](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ipp.html#gs.gxwq5p) Library ([license](https://www.intel.com/content/dam/develop/external/us/en/documents/pdf/intel-simplified-software-license-version-august-2021.pdf)) which can be installed via conda from the `intel` [channel](https://anaconda.org/intel/ipp).
 2. [TIGRE](https://github.com/CERN/TIGRE), which can be installed via conda from the `ccpi` channel.
 
+## Docker
+
+Finally, CIL can be run via a Jupyter Notebook enabled Docker container:
+
+```sh
+docker run --rm --gpus all -p 8888:8888 -it ghcr.io/tomographicimaging/cil:latest
+```
+
+GPU support requires [`nvidia-container-toolkit`](https://github.com/NVIDIA/nvidia-container-toolkit) and an NVIDIA GPU.
+Omit the `--gpus all` to run without GPU support.
+
 # Getting Started with CIL
 
 ## CIL on binder
@@ -74,7 +85,7 @@ git clone --recurse-submodule git@github.com:TomographicImaging/CIL.git
 The use of `--recurse-submodule` is necessary if the user wants the examples data to be fetched (they are needed by the unit tests). We have moved such data, previously hosted in this repo at `Wrappers/Python/data` to the [CIL-data](https://github.com/TomographicImaging/CIL-Data) repository and linked it to this one as submodule. If the data is not available it can be fetched in an already cloned repository as
 
 ```sh
-git submodule update --init
+git submodule update --init --recursive
 ```
 
 ## Build dependencies
@@ -123,6 +134,15 @@ cmake .. -DCMAKE_INSTALL_PREFIX=<install_directory> -DIPP_LIBRARY=<path_to_ipp_l
 
 The user will then need to add the path `<install_directory>/lib` to the environment variable `PATH` or `LD_LIBRARY_PATH`, depending on system OS.
 
+## Building with Docker
+
+In the repository root, simply update submodules and run `docker build`:
+
+```sh
+git submodule update --init --recursive
+docker build . -t ghcr.io/tomographicimaging/cil
+```
+
 # References
 
 [1] Jørgensen JS et al. 2021 [Core Imaging Library Part I: a versatile python framework for tomographic imaging](https://doi.org/10.1098/rsta.2020.0192). Phil. Trans. R. Soc. A 20200192. [**Code.**](https://github.com/TomographicImaging/Paper-2021-RSTA-CIL-Part-I) [Pre-print](https://arxiv.org/abs/2102.04560)

scripts/requirements-test.yml

@@ -17,13 +17,14 @@ channels:
   - conda-forge
   - intel
   - ccpi
-  - astra-toolbox
   - defaults
 dependencies:
-  - python
+  # base (vis. recipe/conda_build_config.yaml)
+  - python=3.10
+  - numpy=1.24
   - cil-data
-  - tigre=2.2
-  - ccpi-regulariser=21.0.0
+  - tigre=2.4
+  - ccpi-regulariser=22.0.0
   - tomophantom=2.0.0
   - astra-toolbox >=1.9.9.dev5,<2.1
   - cvxpy