Create a Sphinx Extension for configuration via YAML

[choldgraf]

In #285 we discussed some ways that Jupyter Book's behavior could be brought into Sphinx. One suggestion was to allow for Jupyter Book's YAML-based configuration in Sphinx.

Currently this is challenging because Sphinx treats conf.py as a special file and so expects it to exist. This is an issue to think about a pattern that might get around this with minimal disruption. Over time, perhaps it can be inspiration for an upstream into Sphinx itself.

1. Create a Sphinx Extension called sphinx-yaml-config. This extension would run early in the build process, and look for a file called either conf.yml or config.yml or _config.yml. If found, it would read in the key/vals as if it was conf.py configuration.

2. The extension would also provide a hook that would be called on the configuration before it was translated to conf.py configuration. This way we could use this hook to do the "Jupyter Book" -> "Sphinx" translation for some config values.

3. When building a Sphinx site, tell users to use the -C option, which tells Sphinx not to use conf.py, along with the the -D option to pass through manual configuration from the CLI.

4. Users would pass a single configuration value, telling Sphinx to activate the YAML extension, something like this:

   sphinx-build -C -D extensions=yaml_config -b html source/ output/

I think that this would then load the YAML config as expected, without Sphinx complaining that no configuration at all was presented.

update: apparently I had already considered doing this a year ago 😆 here's a tiny unfinished prototype of what the tool could look like:

https://github.com/choldgraf/sphinx-yaml-metadata/blob/master/sphinx_yaml_metadata/__init__.py#L8

Unanswered questions

• This would still feel a bit clunky because users would have to manually call sphinx-build. Does anybody see an alternative to this?
• How can we still support the ability to call custom Python from config.yml? In that case do we direct people to also use conf.py? Is there a way to run this code manually during the build process?

[choldgraf]

Ah I think this would actually be harder than expected, because I believe Sphinx only loads extensions upon launch. So you could update almost all of the configuration except for extensions, which Sphinx assumes had already been loaded before then.

[ericholscher]

Yea, I think this might need to be something that is built into Sphinx itself. Might be worth having the discussion in their issue tracker to see if there's much interest if someone wants to do a PR for it.

[choldgraf]

Ah I came to the same conclusion @ericholscher and opened up sphinx-doc/sphinx#9040

Wanna chime in there and see if others are interested?

[choldgraf]

I'm gonna close this one because I don't think it's actionable for us. Fingers crossed that we can get it into sphinx somehow