Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'NoneType' object has no attribute 'argv' with nbconvert 5.4 and nbsphinx #878

Closed
astrojuanlu opened this issue Sep 7, 2018 · 8 comments
Closed

'NoneType' object has no attribute 'argv' with nbconvert 5.4 and nbsphinx #878

astrojuanlu opened this issue Sep 7, 2018 · 8 comments
Milestone
5.5

Comments

@astrojuanlu
Copy link

Recent nbconvert release broke nbsphinx:

$ sphinx-build -vW -b html docs/source docs-html
Running Sphinx v1.7.9
loading pickled environment... failed: build environment version not current
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from http://docs.astropy.org/en/stable/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/numpy/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/scipy/reference/objects.inv...
loading intersphinx inventory from http://matplotlib.org/objects.inv...
intersphinx inventory has moved: http://matplotlib.org/objects.inv -> https://matplotlib.org/objects.inv
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 26 source files that are out of date
updating environment: 26 added, 0 changed, 0 removed
reading sources... [  3%] about
reading sources... [  7%] api
reading sources... [ 11%] changelog
reading sources... [ 15%] core
reading sources... [ 19%] dastcom5 parameters
reading sources... [ 23%] examples/Catch that asteroid!
reading sources... [ 26%] examples/Comparing Hohmann and bielliptic transfers

Traceback (most recent call last):

  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbsphinx.py", line 653, in parse
    rststring, resources = exporter.from_notebook_node(nb, resources)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbsphinx.py", line 575, in from_notebook_node
    nb, resources = pp.preprocess(nb, resources)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 359, in preprocess
    with self.setup_preprocessor(nb, resources, km=km):
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 304, in setup_preprocessor
    self.km, self.kc = self.start_new_kernel(cwd=path)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 253, in start_new_kernel
    km.start_kernel(extra_arguments=self.extra_arguments, **kwargs)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/jupyter_client/manager.py", line 246, in start_kernel
    kernel_cmd = self.format_kernel_cmd(extra_arguments=extra_arguments)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/jupyter_client/manager.py", line 170, in format_kernel_cmd
    cmd = self.kernel_spec.argv + extra_arguments
AttributeError: 'NoneType' object has no attribute 'argv'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/cmdline.py", line 304, in main
    app.build(args.force_all, filenames)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/application.py", line 331, in build
    self.builder.build_update()
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 342, in build_update
    'out of date' % len(to_build))
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 355, in build
    updated_docnames = set(self.env.update(self.config, self.srcdir, self.doctreedir))
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 568, in update
    self._read_serial(docnames, self.app)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 587, in _read_serial
    self.read_doc(docname, app)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 662, in read_doc
    doctree = read_doc(self.app, self, self.doc2path(docname))
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/sphinx/io.py", line 302, in read_doc
    pub.publish()
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/juanlu/.miniconda36/envs/_test36_pol/lib/python3.6/site-packages/nbsphinx.py", line 664, in parse
    str(e))
nbsphinx.NotebookError: AttributeError in examples/Comparing Hohmann and bielliptic transfers.ipynb:
'NoneType' object has no attribute 'argv'

Notebook error:
AttributeError in examples/Comparing Hohmann and bielliptic transfers.ipynb:
'NoneType' object has no attribute 'argv'

If I remove the notebooks and the references to nbsphinx everything works fine. If I downgrade to 5.3.1 and keep the notebooks, everything works too.

I report the issue here because I understand that projects that depend on nbconvert should not break from 5.3 to 5.4, but perhaps it's indeed an nbsphinx issue. cc @spatialaudio

Environment details below:

$ conda info -a

     active environment : _test36_pol
    active env location : /home/juanlu/.miniconda36/envs/_test36_pol
            shell level : 1
       user config file : /home/juanlu/.condarc
 populated config files : /home/juanlu/.miniconda36/.condarc
          conda version : 4.5.11
    conda-build version : 3.12.1
         python version : 3.6.5.final.0
       base environment : /home/juanlu/.miniconda36  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/juanlu/.miniconda36/pkgs
                          /home/juanlu/.conda/pkgs
       envs directories : /home/juanlu/.miniconda36/envs
                          /home/juanlu/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.5 Linux/4.15.0-33-generic linuxmint/18.3 glibc/2.23
                UID:GID : 1000:1000
             netrc file : /home/juanlu/.netrc
           offline mode : False

# conda environments:
#
base                     /home/juanlu/.miniconda36
_numba36_test            /home/juanlu/.miniconda36/envs/_numba36_test
_test27                  /home/juanlu/.miniconda36/envs/_test27
_test36                  /home/juanlu/.miniconda36/envs/_test36
_test36_2                /home/juanlu/.miniconda36/envs/_test36_2
_test36_pol           *  /home/juanlu/.miniconda36/envs/_test36_pol
_testpl27                /home/juanlu/.miniconda36/envs/_testpl27
_testpl36                /home/juanlu/.miniconda36/envs/_testpl36
api36                    /home/juanlu/.miniconda36/envs/api36
astropy36_dev            /home/juanlu/.miniconda36/envs/astropy36_dev
brte36                   /home/juanlu/.miniconda36/envs/brte36
brte36dsk18              /home/juanlu/.miniconda36/envs/brte36dsk18
geostore36               /home/juanlu/.miniconda36/envs/geostore36
gmat36                   /home/juanlu/.miniconda36/envs/gmat36
marvin34                 /home/juanlu/.miniconda36/envs/marvin34
mbd36                    /home/juanlu/.miniconda36/envs/mbd36
pgadmin4                 /home/juanlu/.miniconda36/envs/pgadmin4
poliastro36              /home/juanlu/.miniconda36/envs/poliastro36
poliastro37              /home/juanlu/.miniconda36/envs/poliastro37
py34                     /home/juanlu/.miniconda36/envs/py34
py36                     /home/juanlu/.miniconda36/envs/py36
                         /home/juanlu/.miniconda36/envs/py36/conda-bld/cspice_1529866334362/_build_env
                         /home/juanlu/.miniconda36/envs/py36/conda-bld/cspice_1529866449168/_build_env
                         /home/juanlu/.miniconda36/envs/py36/conda-bld/cspice_1529866543542/_build_env
py37                     /home/juanlu/.miniconda36/envs/py37
py37_def                 /home/juanlu/.miniconda36/envs/py37_def
pypiserver36             /home/juanlu/.miniconda36/envs/pypiserver36
pyrenfe36                /home/juanlu/.miniconda36/envs/pyrenfe36
spice37                  /home/juanlu/.miniconda36/envs/spice37
telluric36               /home/juanlu/.miniconda36/envs/telluric36
unitshack36              /home/juanlu/.miniconda36/envs/unitshack36
                         /home/juanlu/Development/poliastro/benchmarks/env/1e6a43cbb25dbca4e803dce4b31b80e5
                         /home/juanlu/Development/poliastro/benchmarks/env/4ef9678914ff6e01972901ec0de8ad50

sys.version: 3.6.5 | packaged by conda-forge | (defau...
sys.prefix: /home/juanlu/.miniconda36
sys.executable: /home/juanlu/.miniconda36/bin/python
conda location: /home/juanlu/.miniconda36/lib/python3.6/site-packages/conda
conda-build: /home/juanlu/.miniconda36/bin/conda-build
conda-convert: /home/juanlu/.miniconda36/bin/conda-convert
conda-develop: /home/juanlu/.miniconda36/bin/conda-develop
conda-env: /home/juanlu/.miniconda36/bin/conda-env
conda-index: /home/juanlu/.miniconda36/bin/conda-index
conda-inspect: /home/juanlu/.miniconda36/bin/conda-inspect
conda-metapackage: /home/juanlu/.miniconda36/bin/conda-metapackage
conda-render: /home/juanlu/.miniconda36/bin/conda-render
conda-skeleton: /home/juanlu/.miniconda36/bin/conda-skeleton
conda-smithy: /home/juanlu/.miniconda36/bin/conda-smithy
conda-verify: /home/juanlu/.miniconda36/bin/conda-verify
user site dirs: 

CIO_TEST: <not set>
CONDA_DEFAULT_ENV: _test36_pol
CONDA_EXE: /home/juanlu/.miniconda36/bin/conda
CONDA_PREFIX: /home/juanlu/.miniconda36/envs/_test36_pol
CONDA_PROMPT_MODIFIER: (_test36_pol) 
CONDA_PYTHON_EXE: /home/juanlu/.miniconda36/bin/python
CONDA_ROOT: /home/juanlu/.miniconda36
CONDA_SHLVL: 1
DEFAULTS_PATH: /usr/share/gconf/cinnamon.default.path
MANDATORY_PATH: /usr/share/gconf/cinnamon.mandatory.path
PATH: /home/juanlu/.miniconda36/envs/_test36_pol/bin:/home/juanlu/.rbenv/bin:$/home/juanlu/.rbenv/shims:/home/juanlu/.gem/ruby/2.3.0/bin:/home/juanlu/.local/share/umake/bin:/home/juanlu/bin:/home/juanlu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
WINDOWPATH: 8


WARNING: could not import _license.show_info
# try:
# $ conda install -n root _license
$ conda list
# packages in environment at /home/juanlu/.miniconda36/envs/_test36_pol:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.11                    <pip>
asn1crypto                0.24.0                    <pip>
astropy                   3.0.4                     <pip>
astroquery                0.3.8                     <pip>
atomicwrites              1.2.1                     <pip>
attrs                     18.2.0                    <pip>
Babel                     2.6.0                     <pip>
backcall                  0.1.0                     <pip>
beautifulsoup4            4.6.3                     <pip>
bleach                    2.1.4                     <pip>
bzip2                     1.0.6                h470a237_2    conda-forge
ca-certificates           2018.8.24            ha4d7672_0    conda-forge
certifi                   2018.8.24                py36_1    conda-forge
cffi                      1.11.5                    <pip>
chardet                   3.0.4                     <pip>
coverage                  4.5.1                     <pip>
cryptography              2.3.1                     <pip>
cycler                    0.10.0                    <pip>
decorator                 4.3.0                     <pip>
defusedxml                0.5.0                     <pip>
docutils                  0.14                      <pip>
entrypoints               0.2.3                     <pip>
gmp                       6.1.2                hfc679d8_0    conda-forge
html5lib                  1.0.1                     <pip>
idna                      2.7                       <pip>
imagesize                 1.1.0                     <pip>
ipykernel                 4.9.0                     <pip>
ipython                   6.5.0                     <pip>
ipython-genutils          0.2.0                     <pip>
ipywidgets                7.4.1                     <pip>
jedi                      0.12.1                    <pip>
jeepney                   0.3.1                     <pip>
Jinja2                    2.10                      <pip>
jplephem                  2.8                       <pip>
jsonschema                2.6.0                     <pip>
jupyter-client            5.2.3                     <pip>
jupyter-core              4.4.0                     <pip>
keyring                   15.0.0                    <pip>
kiwisolver                1.0.1                     <pip>
libffi                    3.2.1                hfc679d8_5    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3    conda-forge
llvmlite                  0.24.0                    <pip>
MarkupSafe                1.0                       <pip>
matplotlib                2.2.3                     <pip>
mistune                   0.8.3                     <pip>
more-itertools            4.3.0                     <pip>
nbconvert                 5.4.0                     <pip>
nbformat                  4.4.0                     <pip>
ncurses                   6.1                  hfc679d8_1    conda-forge
notebook                  5.6.0                     <pip>
numba                     0.39.0                    <pip>
numpy                     1.15.1                    <pip>
openssl                   1.0.2p               h470a237_0    conda-forge
packaging                 17.1                      <pip>
pandas                    0.23.4                    <pip>
pandoc                    1.19.2                        0    conda-forge
pandocfilters             1.4.2                     <pip>
parso                     0.3.1                     <pip>
pexpect                   4.6.0                     <pip>
pickleshare               0.7.4                     <pip>
pip                       18.0                     py36_1    conda-forge
plotly                    3.2.0                     <pip>
pluggy                    0.7.1                     <pip>
poliastro                 0.11.dev0                 <pip>
prometheus-client         0.3.1                     <pip>
prompt-toolkit            1.0.15                    <pip>
ptyprocess                0.6.0                     <pip>
py                        1.6.0                     <pip>
pycodestyle               2.4.0                     <pip>
pycparser                 2.18                      <pip>
Pygments                  2.2.0                     <pip>
pyparsing                 2.2.0                     <pip>
pytest                    3.8.0                     <pip>
pytest-cov                2.6.0                     <pip>
python                    3.6.6                h5001a0f_0    conda-forge
python-dateutil           2.7.3                     <pip>
pytz                      2018.5                    <pip>
pyzmq                     17.1.2                    <pip>
readline                  7.0                  haf1bffa_1    conda-forge
requests                  2.19.1                    <pip>
retrying                  1.3.3                     <pip>
scipy                     1.1.0                     <pip>
SecretStorage             3.1.0                     <pip>
Send2Trash                1.5.0                     <pip>
setuptools                40.2.0                   py36_0    conda-forge
simplegeneric             0.8.1                     <pip>
six                       1.11.0                    <pip>
snowballstemmer           1.2.1                     <pip>
Sphinx                    1.7.9                     <pip>
sphinx-rtd-theme          0.4.1                     <pip>
sphinxcontrib-websupport  1.1.0                     <pip>
sqlite                    3.24.0               h2f33b56_1    conda-forge
terminado                 0.8.1                     <pip>
testpath                  0.3.1                     <pip>
tk                        8.6.8                         0    conda-forge
tornado                   5.1                       <pip>
traitlets                 4.3.2                     <pip>
urllib3                   1.23                      <pip>
wcwidth                   0.1.7                     <pip>
webencodings              0.5.1                     <pip>
wheel                     0.31.1                   py36_1    conda-forge
widgetsnbextension        3.4.1                     <pip>
xz                        5.2.4                h470a237_1    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge
astrojuanlu added a commit to astrojuanlu/poliastro that referenced this issue Sep 7, 2018
@astrojuanlu
Restrict nbconvert version
2b0b082 
See jupyter/nbconvert#878
@akhmerov
Copy link
Member

akhmerov commented Sep 7, 2018

Can you try to narrow down the issue? From looking at the code I see that:

  • The failure is because the execute preprocessor is not instantiated with a kernel spec
  • nbsphinx has a default kernel name "", which should make execute preprocessor look into what's stored in the notebook
  • the notebook has the kernel correctly specified

Therefore it appears that the problem shouldn't be happening. Also looking at the changes in the ExecutePreprocessor I cannot figure out what could be triggering the problem.

How are you calling nbsphinx? Can you figure out what part of the code specifies the kernel spec to be equal to None?

@astrojuanlu
Copy link
Author

Sorry, I meant to ping @mgeier above.

Thanks for your quick answer @akhmerov! I am "just" a nbsphinx user so I don't know those details right away. If Matthias doesn't chime in, I will try to lend a hand in about a couple of weeks.

@mgeier
Copy link
Contributor

mgeier commented Sep 8, 2018

As a work-around you could try setting this:

nbsphinx_kernel_name = 'python3'

But normally, this shouldn't be necessary.

@astrojuanlu astrojuanlu mentioned this issue Sep 8, 2018
Open
@mgeier
Copy link
Contributor

mgeier commented Sep 9, 2018

I just updated my nbconvert installation and now I'm also getting the error!

I bisected into it and found that the error was introduced in #816, and more exactly in commit c1cb89c, committted by @mpacer.

@mgeier mgeier mentioned this issue Sep 9, 2018
Merged
@Carreau Carreau added this to the 5.5 milestone Sep 10, 2018
@mgeier mgeier mentioned this issue Sep 10, 2018
Closed
drasmuss added a commit to nengo/nengo-dl that referenced this issue Sep 10, 2018
@drasmuss
Fix nbconvert/nbsphinx conflict in doc build
6cff5af 
See jupyter/nbconvert#878
@mpacer
Copy link
Member

mpacer commented Sep 11, 2018

I think this may be due to nbspbinx specifying the kernel directly by setting it equal to “”. That would override the default value (where the kernel name is now being grabbed from the notebook dynamically) and as a result it never hits the code that would have grabbed it from the notebook.

We can add that behaviour back in since we had it before. But it does seem like a previous anti pattern to require specifying an option in order to trigger what should be default behaviour (adhering to the notebook’s kernel name).

This is speculation based on looking at the code based I’ll need to test this out in a bit.

drasmuss added a commit to nengo/nengo-dl that referenced this issue Sep 11, 2018
@drasmuss
Fix nbconvert/nbsphinx conflict in doc build
f44a115 
See jupyter/nbconvert#878
@mpacer mpacer mentioned this issue Sep 11, 2018
Closed
@mgeier
Copy link
Contributor

mgeier commented Sep 18, 2018

I've disabled nbconvert release 5.4 in the latest nbsphinx release, so updating nbsphinx should avoid the error for now.

I hope the actual problem will be fixed in #886 before the next nbconvert release.

@astrojuanlu astrojuanlu mentioned this issue Sep 23, 2018
Closed
timstaley added a commit to timstaley/voevent-parse that referenced this issue Sep 29, 2018
@timstaley
Semi-Pin nbconvert to avoid this issue: jupyter/nbconvert#878
58fc1eb
@defjaf
Copy link

defjaf commented Oct 24, 2018

So, is it better to be using the latest nbsphinx with an older nbconvert, or vice versa?

@mgeier
Copy link
Contributor

mgeier commented Oct 24, 2018

When you install the latest nbsphinx release, the conflicting nbconvert version 5.4 should be disabled automatically (see spatialaudio/nbsphinx#206). If not, you can try to use nbconvert!=5.4 manually.

The other way round it will not work, every version of nbsphinx is broken when using nbconvert 5.4.

I hope PR #886 will be solved before the next nbconvert release.

@mgeier mgeier mentioned this issue Nov 7, 2018
Open
@mgeier mgeier mentioned this issue Dec 4, 2018
Closed
mgeier added a commit to mgeier/nbconvert that referenced this issue Dec 4, 2018
@mgeier
ExecutePreprocessor: Get kernel name when executing
d90026f 
... and not during initialization.

Fixes jupyter#878.
@mgeier mgeier mentioned this issue Dec 4, 2018
Merged
@MSeal MSeal closed this as completed in ad27a93 Dec 7, 2018
@mgeier mgeier mentioned this issue Dec 8, 2018
Closed
@astrojuanlu astrojuanlu mentioned this issue Dec 26, 2018
Merged
andportnoy added a commit to kutaslab/fitgrid that referenced this issue Feb 4, 2019
@andportnoy
Restrict nbconvert version to avoid version 5.4
3650953 
This causes nbsphinx failures, for more details see:

spatialaudio/nbsphinx#202
jupyter/nbconvert#878
jupyter/nbconvert#886

Should be fixed by the next nbconvert release (5.5).
@andportnoy andportnoy mentioned this issue Feb 4, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
5.5
Development

Successfully merging a pull request may close this issue.

WIP Recreate old behaviour of ExecutePreprocessor(kernel_name="") mpacer/nbconvert
6 participants
@astrojuanlu @Carreau @mgeier @defjaf @akhmerov @mpacer