Add ROOT_CONTAINER to base-notebook

[guoquan]

Add another layer of abstraction to the base container, namely the root container ROOT_CONTAINER, so that the user can have the flexibility to build the whole series of images with make based on a different base image.

For example, one can build a CUDA stack, though you guy might not like this example, by:

DARGS="--build-arg ROOT_CONTAINER=nvcr.io/nvidia/cuda:10.2-runtime-ubuntu18.04" make build-all

[parente]

Thanks @guoquan. This makes sense to me.

It might be nice to add it to the documentation so people know it's available and how to use it. We shouldn't hold this PR for that: you or someone else looking to contribute can send a separate PR with the documentation changes.

[maresb]

Ah, now I see it!!!

I was confused why ROOT_CONTAINER was necessary when we already have BASE_CONTAINER. For slow people like me, ROOT_CONTAINER is necessary because BASE_CONTAINER exists in each of the notebooks, not just base-notebook. If one sets BASE_CONTAINER and tries to build tensorflow-notebook, then Docker will build tensorflow-notebook directly on top of CUDA, completely ignoring base-notebook through scipy-notebook. Instead, we want tensorflow-notebook to build on top of the full chain from scipy-notebook, all the way down to base-notebook, and only then on CUDA.

This is the solution to the sub-issue raised in #1015.

[ChristophSchranz]

Hi, I was looking for a solution like that myself, but was too stupid to get this, so I've created a script that concatenates, see #1015
If i run inside tensorflow-notebook

docker build --build-arg ROOT_CONTAINER=nvidia/cuda:10.2-base-ubuntu18.04 -t notebook .

this will pull the standard image from jupyter/scipy-notebook with the old BASE_CONTAINER. How does it work on your system?

[maresb]

Ah, right, ROOT_CONTAINER is only defined in base-notebook. Thus if base-notebook is already built and cached, then the ROOT_CONTAINER build arg has no effect. For this ROOT_CONTAINER argument to work as expected, it needs to be recursive. In other words, each Dockerfile's FROM should somehow pass along the ROOT_CONTAINER argument.

Having a quick look at the Dockerfile reference for FROM and ARG, my first thought is that maybe we could pass ROOT_CONTAINER as a tag, i.e. something like jupyter/scipy-notebook:root_nvidia/cuda:10.2-base-ubuntu18.04. Unfortunately I don't have any more time at the moment to think through this problem properly.

[guoquan]

docker build --build-arg ROOT_CONTAINER=nvidia/cuda:10.2-base-ubuntu18.04 -t notebook .

It seems this would not work as expected because it will pull from either the registry (which is not CUDA) or cache (which might not be CUDA as @maresb mentioned). I do not have any idea if docker build has such a mechanism to rebuild parent base on ARG. My quick thought would be relying on makefile to maintain the dependency and force rebuild locally. It could be possible to avoid some rebuild by setting and checking LABEL for ROOT_CONTAINER.