Env variable for extra_template_basedirs

[govinda18]

Hi team,

I want to be able to set extra_template_basedirs via some env variable. I am using in an internal infrastructure that needs this and some external library seems to use PythonExporter to convert a notebook into python.

Would be helpful to set this globally so this gets automatically picked up.

[Carreau]

I'd like to understand this request a bit more instead of providing a band aid for a single option.

Do you have a way of creating ~/.jupyter/jupyter_nbconvert_config.py (or any other default config file) ? You should be able to also find all the places you can add those default configs using jupyter --paths.

With this you should be able to hook into any logics you want without waiting for a patch/release.

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

extra = os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
if extra:
    # you can even use `.append()` or `.extend()` on `c.Exporter.extra_template_basedirs`
    c.Exporter.extra_template_basedirs = [
        os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
    ]

Or maybe the software you use explicitly as nbconvert to not read its config, and hence the need for a patch.

This does not means we cannot add the option regardless, but using the default config capabilities should allow you to modify any config option instead of creating and add-hoc version every time.

If there is a global reason to use ENV variable instead of doing on a per-use-case, we could imaging having a mapping from env variable to configuration option, say NBCONVERT__<classname>__<Option_name>=value.

That would make sens I think, but you also should be able to achieve it now with:

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

for k, v in os.environ.items():
    if k.startswith("NBCONVERT"):
        # use __ instead of . as separator in env variable.
        # Warning, case sensitive !
        _, klass, attribute = k.split("__")
        # c.Klass.attribute=v
        setattr(getattr(c, klass), attribute, v)

Allows me to set $ export NBCONVERT__Exporter__extra_template_basedirs='mypath', and this will have the expected effect.

[ndmlny-qs]

Thanks for the thoughtful considerations @Carreau. @govinda18 can chime in on their use case, but I can think of one where environment variables would be useful.

If you have an automated deploy scenario for clients, and want all of them to use a centralized extra nbconvert templates directory that you have custom artifacts in, then I see two options available:

1. in your deploy scripts, you generate the jupyter_nbconvert_config.py file for each client that gets spun up;
2. have an env variable that nbconvert recognizes on each client.

I suspect that (since it is easy to add/remove environment variables for newly create clients) having nbconvert read environment variables compared to updating client preferences in a python file is the easiest change from a devops perspective. This is of course is just a guess.

[Carreau]

Yes, this is my guess as well, but if that's the case then likely the need is likely not limited to nbconvert, and it might be worth it to patch traitlets to load from env by default in the same way that we have a config file loader and and cli loader ?

[Carreau]

@govinda18 I wanted to know if you had a chance to look at my suggestions and clarify your use case.

[Carreau]

#2075 might help