ReadtheDocs failure with version 4.9.0

[bhilbert4]

Hello,

The ReadTheDocs builds for my package began failing about an hour ago, and it looks like the cause is the release of version 4.9.0. At least, that's the only difference between earlier passing builds and the new failing builds. Some googling showed me that Jupyterlab had a similar error a while ago, but it's not at all clear to me if or how that fix would be applicable in this case:
jupyterlab/jupyterlab#3125

The full log of the failing RTD build is here:
https://readthedocs.org/api/v2/build/15101113.txt

And here is an excerpt from the log with the traceback:

[rtd-command-info] start-time: 2021-10-27T00:35:39.350517Z, end-time: 2021-10-27T00:35:40.029802Z, duration: 0, exit-code: 2
/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/bin/python -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
Running Sphinx v4.2.0
loading translations [en]... done

Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/sphinx/application.py", line 237, in init
self.setup_extension(extension)
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/sphinx/application.py", line 393, in setup_extension
self.registry.load_extension(self, extname)
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/sphinx/registry.py", line 429, in load_extension
mod = import_module(extname)
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 783, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbsphinx.py", line 42, in
import nbconvert
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbconvert/init.py", line 4, in
from .exporters import *
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbconvert/exporters/init.py", line 3, in
from .html import HTMLExporter
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbconvert/exporters/html.py", line 26, in
from .templateexporter import TemplateExporter
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 122, in
class TemplateExporter(Exporter):
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 246, in TemplateExporter
jupyter_path('nbconvert','templates'),
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/jupyter_core/paths.py", line 171, in jupyter_path
userdir = os.path.join(site.getuserbase(), 'share', 'jupyter')
AttributeError: module 'site' has no attribute 'getuserbase'

Exception occurred:
File "/home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743/lib/python3.8/site-packages/jupyter_core/paths.py", line 171, in jupyter_path
userdir = os.path.join(site.getuserbase(), 'share', 'jupyter')
AttributeError: module 'site' has no attribute 'getuserbase'
The full traceback has been saved in /tmp/sphinx-err-2_2htboj.log, if you want to report the issue to the developers.

[jasongrout]

Very interesting. According to https://docs.python.org/3/library/site.html#site.getuserbase, this function was introduced in Python 3.2, and it appears from the traceback it is using python 3.8.

[jasongrout]

According to the jlab issue (which is years old at this point, and is mainly about Python 2.7), there may be an issue with virtualenv? pypa/virtualenv#355

[jasongrout]

Interestingly, one of the first lines of your logs is:

python3.8 -mvirtualenv --system-site-packages /home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743

Perhaps it is still a problem with virtualenv, even in Python 3.8?

[jasongrout]

More context (again, from a while ago, but perhaps still relevant?): pypa/virtualenv#737 (comment)

It appears that an updated virtualenv package may solve the issue? alexa/ask-toolkit-for-vscode#27 (comment)

[bhilbert4]

Thanks for the investigation, and the fix. Just for completeness, I tried having RTD install the latest version of virtualenv, but since the command you pointed out above:

python3.8 -mvirtualenv --system-site-packages /home/docs/checkouts/readthedocs.org/user_builds/mirage-data-simulator/envs/743

happens before RTD installs packages in the environment, it didn't help:
https://readthedocs.org/api/v2/build/15106696.txt

[jasongrout]

I just released this fix in 4.9.1rc0. Can one of you (@bhilbert4, @jakevdp) test to see if 4.9.1rc0 fixes your RTD build? If it does, I'll go ahead and release 4.9.1 with this fix.

[jasongrout]

I also filed a support request with ReadTheDocs to ask about upgrading virtualenv in the base image.

[bhilbert4]

4.9.1rc0 looks like it solved the problem for me. RTD build completed successfully. Thanks!!

[jakevdp]

Confirmed in jax-ml/jax#8383 that 4.9.1rc0 fixes the issue (see RTD build log). Thanks!

[astrojuanlu]

Hi! Arrived here from the RTD email, posting here for broader reach. Does upgrading to our latest build images help? Essentially this:

diff --git a/.readthedocs.yml b/.readthedocs.yml
index a9815054..dcfea8c4 100644
--- a/.readthedocs.yml
+++ b/.readthedocs.yml
@@ -5,6 +5,11 @@
 # Required
 version: 2
 
+build:
+  os: "ubuntu-20.04"
+  tools:
+    python: "3.8"
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
   configuration: docs/conf.py
@@ -15,7 +20,6 @@ formats:
 
 # Optionally set the version of Python and requirements required to build your docs
 python:
-  version: 3.8
   install:
     - method: pip
       path: .

(sent spacetelescope/mirage#745 as a test)

[jakevdp]

Confirmed that upgrading to the latest build images works with jupyter-core v4.9.0 (build log).

[bhilbert4]

This fix also works in spacetelescope/mirage#745 with jupyter-core 4.9.0. Thanks!

[jasongrout]

Hi! Arrived here from the RTD email, posting here for broader reach. Does upgrading to our latest build images help?

Thanks so much for posting a solution and seeking out where to post it for high visibility. Thanks!

[jasongrout]

I just released 4.9.1 on pypi for people using older virtualenv, though I agree that the better fix for these readthedocs issues is to upgrade the base image.