Use conda on RTD

[minrk]

continuation of #184

[takluyver]

Neat, I didn't know that this was possible.

[takluyver]

Now, unfortunately, it has merge conflicts.

Have you tested this on RTD?

[minrk]

Waiting for RTD build now, I'll ping back when I'm sure it works.

[takluyver]

Did the RTD build go OK?

[minrk]

Sorry, lost track. It did not, looking into why, now...

[minrk]

Lame, pandoc seems to not be loading libgmp...

[minrk]

Marking back as WIP, since there always seems to be something going awry.

[minrk]

Part of the problem seems to be caused by nbsphinx executing notebooks, which we don't want to happen on RTD. Is there a way to prevent that?

[minrk]

It works!

I just had to run locally and commit the output, to avoid nbsphinx trying to run it on RTD.

One weird bit was that the pandoc conda package doesn't seem to work with python 3, due to a missing libgmp, but it works fine with Python 2. No idea why that would be, but it works now.

[takluyver]

I thought nbsphinx removed the need for pandoc to do this?

[mgeier]

The main point for using nbsphinx was to avoid committing the notebook results to the repo and instead running the notebooks on-the-fly on readthedocs.org.

AFAICT, pandoc is still used via the Jinja filter markdown2rst, right?

[takluyver]

Ah, right, I didn't realise that nbsphinx was converting notebooks to rst as an intermediate stage - I thought it was loading them directly into Sphinx, similar to what recommonmark (presumably) does for Markdown.

[mgeier]

OK, I tried this conda thing on readthedocs, but it didn't work and reported some errors.

Then I tried it with plain old requirements.txt (even removing most of them, but temporarily adding entrypoints) and it worked perfectly: http://nbconvert-conda-experiment.readthedocs.org/en/requirements.txt/
Including executing the notebooks and everything.

Isn't this good enough? Why do you even want to switch to conda?

BTW, the dependency on entrypoints in requirements.txt will only be necessary for building the latest version of the docs until the next release of nbconvert on PyPI. For now, it's needed because of your policy to not use setuptools.

[takluyver]

That's the same error I got:

Error: ERROR: placeholder '/root/miniconda3/envs/_build_placehold_placehold_placehold_placehold_placehold_p' too short in: ncurses-5.9-5

I think it comes about when the environment path, which includes the project name on RTD and the git branch name, is too long. So it should be fine in the main repo. See readthedocs/readthedocs.org#1902 for more details.

If nbsphinx needs pandoc, how is it working with requirements.txt?

[mgeier]

I thought it was loading them directly into Sphinx, similar to what recommonmark (presumably) does for Markdown.

Initially, I wanted to convert directly to the docutils format, but I didn't have enough experience then.
So I just used the tools that were already available in nbconvert and took the detour via reST.

At some point, I might consider re-implementing it without the detour, but currently that's not my plan.

If nbsphinx needs pandoc, how is it working with requirements.txt?

I guess because pandoc is already installed on readthedocs?
http://packages.readthedocs.org/en/latest/deb/

[takluyver]

Huh, I guess it makes sense that pandoc is there, but I'm surprised we didn't spot it before.

[minrk]

Any reason not to merge, then?

[takluyver]

Let's find out!

[mgeier]

I was a little bit too late to object ...

I don't think that it's a good idea to add the manually created notebook outputs after removing the manually created *.rst files in #212.

The notebooks can be auto-executed on readthedocs perfectly fine without conda and without the need to commit any auto-generated bloat to the repository. As I've shown above.

[minrk]

The notebooks can be auto-executed on readthedocs perfectly fine

I added the outputs specifically because this wasn't working, and I don't think we want to be executing notebooks on RTD.

[mgeier]

I added the outputs specifically because this wasn't working

Well it is working perfectly without conda, see #222 (comment).

I don't think we want to be executing notebooks on RTD

OK, fair enough, but might I ask why?

I think it would be great to execute the notebooks on RTD, because then it is assured that the documentation is up-to-date with the implementation.
When saving the notebook outputs, one has to make sure to re-generate and re-commit them (manually) whenever any output potentially changes. In addition to being potentially out-ouf-date, this gradually adds bloat to the repository.

But probably I'm missing some disadvantages of executing them on RTD?