Support/ideas for a gallery

[maartenbreddels]

I'm thinking/working on getting a gallery but for ipywidgets (ipyvolume, bqplot, ipyleaflet), ala sphinx-gallery in sphinx-gallery/sphinx-gallery#422
Sphinx-gallery however, is script oriented. All examples are in scripts, and multiple sections are separated by special markers.

Since I'd like it to work with a notebook, and nbsphinx already does the notebook -> rst transformation, I wonder if you though about supporting this, or how to build on top of this.

I am not interested in creating a new package, but more for improving existing packages and at the most make a really thin layer to combine it all.

Maybe an idea would be to have a special directive, that takes as argument a glob pattern of notebook files, these will be passed to nbsphinx, and (somehow) the output images are used to create a gallery possibly leveraging some sphinx-gallery code.

Concrete questions

• Can I invoke nbsphinx manually, like pass it a notebook, or let it take care of the rst generation, do you know of examples, or could you point me in the right direction?
• Would there be some 'hook' to find which images (png) are created so I can find out which images are produced by which notebook, so a gallery can be build.
• Did you think about this, or are you interested in this?
• Would you consider reviewing PR's that would enable some of this?

[mgeier]

Thanks for the suggestion! I haven't really thought about this yet. My main interest is using notebooks as Sphinx source files and preserving as many features as possible between the notebook view and the HTML and LaTeX output of Sphinx.

I could imagine a gallery-like thing that acts as kind of an overview of the following (group of) notebooks. This should probably internally have something like a toctree directive with the "hidden" option. It would in fact probably be very similar to the toctree directive (including globbing and stuff).
I would start with something that can be created in a .rst file, but if that works, we could also think about supporting it in a notebook, similar to https://nbsphinx.readthedocs.io/en/0.3.5/subdir/toctree.html.

I wouldn't rely on the internal reST representation though, because at some point I would like to get rid of it, see #36.

It's better to use the in-memory docutils document representation.

Can I invoke nbsphinx manually, like pass it a notebook, or let it take care of the rst generation, do you know of examples, or could you point me in the right direction?

Yes, that should be possible, but only until #36 is done:

import nbsphinx
rst, res = nbsphinx.Exporter().from_filename('my-notebook.ipynb')
print(rst)

You can also check out the save-rst branch, which saves the intermediate *.rst files to the HTML output directory.

But again, you shouldn't rely on the intermediate reST representation.

Would there be some 'hook' to find which images (png) are created so I can find out which images are produced by which notebook, so a gallery can be build.

There isn't really a "hook". The cell outputs are stored in the resources variable after the notebook has been executed.

Here they are written to files:

nbsphinx/src/nbsphinx.py

Lines 836 to 841 in 27a38ca

	# Create additional output files (figures etc.),
	# see nbconvert.writers.FilesWriter.write()
	for filename, data in resources.get('outputs', {}).items():
	dest = os.path.normpath(os.path.join(srcdir, filename))
	with open(dest, 'wb') as f:
	f.write(data)

The file names contain the name of the notebook (with slashes and spaces replaced by underscores), followed by the cell number and the output number (because there might be more than one output per cell). The base name is specified here:

nbsphinx/src/nbsphinx.py

Line 811 in 27a38ca

resources['unique_key'] = re.sub('[/ ]', '_', env.docname)

Note that the order in which Sphinx source files are parsed is kinda arbitrary. You would have to store a placeholder in the document where the gallery should be and then, when all source files are processed, you can create the actual content of the gallery.

But this is in fact very similar to the toctree directive, which displays the section titles, which are only known after parsing the source files.

I think the Sphinx extension tutorial also describes a similar situation: http://www.sphinx-doc.org/en/master/extdev/tutorial.html.

Did you think about this, or are you interested in this?

Not really. Yes.

Would you consider reviewing PR's that would enable some of this?

Definitely, this would be great!

[mgeier]

Just to be sure: you know sphinx-nbexamples, right?

https://sphinx-nbexamples.readthedocs.io/en/dev/

https://github.com/Chilipp/sphinx-nbexamples

Maybe that already does what you want?

I haven't really looked into that yet ...

[mgeier]

@maartenbreddels Just a little heads up: I'm currently working on a gallery feature for nbsphinx in this branch: https://github.com/mgeier/nbsphinx/tree/gallery

I still need to figure out a few things, but then I'll make a PR soon-ish.

[mgeier]

I've just made a WIP PR: #392

[mgeier]

#392 is now ready for review!

[maartenbreddels]

Awesome work!