Cascading Schemas

[mlucool]

Problem

Jupyter's config (and data) use a cascading model for loading configs. An admin can set a systemwide default config in the venv which overrides the Jupyter defaults; then a user can set another config to override those. This model works very well for deploying Jupyter in a shared environment. Schemas don't share this cascading model, making it hard to override defaults as an administrator. In jupyter@2.0, an admin could override the generated schema directory, but with federated modules, that does not seem possible.

Proposed Solution

Add a third directory for schemas that is configurable. This would be merged with federated/installed modules default schema to become the "System Defaults" that a user sees. "User preferences" overriding these would remain unchanged.

Note: I am unfamiliar with this code, so there may be a much simpler way.

[jasongrout]

You can still override schema defaults as a system administrator with the overrides.json file. See https://jupyterlab.readthedocs.io/en/stable/user/directories.html?highlight=overrides.json#overrides-json

[mlucool]

Hi @jasongrout,

This worked pretty well, but I ran into three issues:

1. From what I can tell there is no way to override shortcuts. Notably, we prefer documentsearch:start as Accel Shift F as you have suggested elsewhere.
2. If you are setting one property of an object, the settings menu loses the other properties. It is understandable why this was chosen, but it means that every new property requires admins to track and add back. Example:

    "@jupyterlab/notebook-extension:tracker": {
        "codeCellConfig": {
            "lineNumbers": True,
          }
     }

3. While minor, it would be nice to support json5 or a py file (mostly so that we can comment why something was done)

[jasongrout]

From what I can tell there is no way to override shortcuts. Notably, we prefer documentsearch:start as Accel Shift F as you have suggested elsewhere.

This is #9858 (for 3.1)

If you are setting one property of an object, the settings menu loses the other properties. It is understandable why this was chosen, but it means that every new property requires admins to track and add back. Example:

Right, we don't merge keys. Perhaps we should? That comes with its own set of problems to make sure things are consistent, etc.

[mlucool]

Perhaps we should? That comes with its own set of problems to make sure things are consistent, etc.

What if we did something a bit hacky?

    "@jupyterlab/notebook-extension:tracker": {
        "codeCellConfig": {
            "__merge": True # Not sure if the default is to merge or replace.
            "lineNumbers": True,
          }
     }

[jasongrout]

What if we did something a bit hacky?

Great question, thanks for asking. I haven't thought a lot about it, so I'm not sure how I feel about it (and of course would love others to chime in as well about what they think). Is this something you'd want to open a PR for?

[mlucool]

Is this something you'd want to open a PR for?

Depends on how complex it would be - any pointers?

Also, where is overrides.json loaded from? Would it be ok if I made a PR to support another format (json5)? Notably, I'd like to have comments in this file. We may want to consider that support for all schema files

[jasongrout]

The code is in https://github.com/jupyterlab/jupyterlab_server/blob/master/jupyterlab_server/settings_handler.py (search for "overrides" to see places where the logic is implemented).

[jasongrout]

Here is where the overrides.json file is actually read in: https://github.com/jupyterlab/jupyterlab_server/blob/34ce153057af9ffdaca8a97203dd554d3c395758/jupyterlab_server/settings_handler.py#L258-L273

[mlucool]

Is this something you'd want to open a PR for?

We can open a PR for this. Any objections to the magic private key proposal above (i.e. __merge)? For backwards compatibility, it should default to not merge. On the other hand, I suspect people expect it to merge as that's what is happening on the top level (e.g. the codeCellConfig is merged into the object as a replacement and we don't lose all other keys)

[mlucool]

@blink1073 this issue is still open as we discuss what to do about _merge. It won't let me reopen, so will you?

[blink1073]

👍🏼

[goanpeca]

We can open a PR for this. Any objections to the magic private key proposal above (i.e. __merge)? For backwards compatibility, it should default to not merge. On the other hand, I suspect people expect it to merge as that's what is happening on the top level (e.g. the codeCellConfig is merged into the object as a replacement and we don't lose all other keys)

Thoughts on this @blink1073 ?