Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 6.1.13 nbconvert failing with TypeError: 'coroutine' object is not subscriptable #637

Open
mriedem opened this issue Apr 7, 2021 · 30 comments

Comments

@mriedem
Copy link

mriedem commented Apr 7, 2021

We have a CI build that runs this notebook:

https://github.com/Qiskit/qiskit-tutorials/blob/stable/0.23.x/tutorials/circuits/1_getting_started_with_qiskit.ipynb

It started failing this morning with this error:

[NbConvertApp] Converting notebook /tmp/qiskit-iqx-tutorials/qiskit/tutorials/circuits/1_getting_started_with_qiskit.ipynb to html
[NbConvertApp] Executing notebook with kernel: python3
Traceback (most recent call last):
  File "/opt/conda/bin/jupyter-nbconvert", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.8/site-packages/jupyter_core/application.py", line 254, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/traitlets/config/application.py", line 837, in launch_instance
    app.start()
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 340, in start
    self.convert_notebooks()
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 510, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 481, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 197, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/html.py", line 95, in from_notebook_node
    return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 307, in from_notebook_node
    nb_copy, resources = super(TemplateExporter, self).from_notebook_node(nb, resources, **kw)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 139, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/exporters/exporter.py", line 316, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 438, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 578, in run_cell
    exec_reply = self._poll_for_reply(parent_msg_id, cell, timeout)
  File "/opt/conda/lib/python3.8/site-packages/nbconvert/preprocessors/execute.py", line 479, in _poll_for_reply
    if msg['parent_header'].get('msg_id') == msg_id:
TypeError: 'coroutine' object is not subscriptable
sys:1: RuntimeWarning: coroutine 'ZMQSocketChannel.get_msg' was never awaited

The only difference in the package lists was jupyter-client 6.1.13 (note that nbconvert is pinned to 5.6.1 in our case).

I don't know how all of this is related, but was wondering if #623 might be causing some issue.

I have excluded the jupyter-client 6.1.13 version from our build and things are working again. My concern is if this comes back in the next release (6.2?).

@mriedem
Copy link
Author

mriedem commented Apr 7, 2021

This is the (pip) package list with a successful build:

MarkupSafe-1.1.1
Send2Trash-1.5.0
argon2-cffi-20.1.0
async-generator-1.10
attrs-20.3.0
backcall-0.2.0
bleach-3.3.0
certifi-2020.12.5
cffi-1.14.5
chardet-4.0.0
colorcet-2.0.6
cryptography-3.4.7
cvxpy-1.1.11
cycler-0.10.0
decorator-4.4.2
defusedxml-0.7.1
dill-0.3.3
dlx-1.0.4
docplex-2.20.204
ecos-2.0.7.post1
entrypoints-0.3
fastdtw-0.3.4
fastjsonschema-2.15.0
h5py-3.1.0
ibm-quantum-widgets-1.0.3
idna-2.10
inflection-0.5.1
ipykernel-5.5.3
ipython-7.22.0
ipython-genutils-0.2.0
ipyvue-1.5.0
ipyvuetify-1.6.2
ipywidgets-7.6.3
iqx-0.1.17
jedi-0.18.0
jinja2-2.11.3
joblib-1.0.1
json-logging-1.3.0
jsonschema-3.2.0
jupyter-1.0.0
jupyter-client-6.1.12
jupyter-console-6.4.0
jupyter-core-4.7.1
jupyterlab-pygments-0.1.2
jupyterlab-widgets-1.0.0
kaleidoscope-0.0.12
kiwisolver-1.3.1
llvmlite-0.36.0
lxml-4.6.3
matplotlib-3.4.1
mistune-0.8.4
more-itertools-8.7.0
mpmath-1.2.1
multitasking-0.0.9
nbclient-0.5.3
nbconvert-6.0.7
nbformat-5.1.3
nbresuse-0.4.0
nest-asyncio-1.5.1
networkx-2.5.1
notebook-6.3.0
ntlm-auth-1.5.0
numba-0.53.1
numpy-1.20.1
osqp-0.6.2.post0
packaging-20.9
pandas-1.2.3
pandocfilters-1.4.3
param-1.10.1
parso-0.8.2
pexpect-4.8.0
pickleshare-0.7.5
pillow-8.2.0
plotly-4.14.3
ply-3.11
prometheus-client-0.10.0
prompt-toolkit-3.0.18
psutil-5.8.0
ptyprocess-0.7.0
pybind11-2.6.2
pycparser-2.20
pyct-0.4.8
pydot-1.4.2
pygments-2.8.1
pylatexenc-2.10
pyparsing-2.4.7
pyperclip-1.8.2
pyrsistent-0.17.3
pyscf-1.7.3
python-constraint-1.4.0
python-dateutil-2.8.1
pytz-2021.1
pyzmq-22.0.3
qdldl-0.1.5.post0
qiskit-0.25.0
qiskit-aer-0.8.0
qiskit-aqua-0.9.0
qiskit-finance-0.1.0
qiskit-ibmq-provider-0.12.2
qiskit-ignis-0.6.0
qiskit-machine-learning-0.1.0
qiskit-nature-0.1.0
qiskit-optimization-0.1.0
qiskit-terra-0.17.0
qtconsole-5.0.3
qtpy-1.9.0
quandl-3.6.0
requests-2.25.1
requests-ntlm-1.1.0
retrying-1.3.3
retworkx-0.8.0
scikit-learn-0.24.1
scipy-1.6.1
scs-2.1.2
seaborn-0.11.1
setuptools-54.2.0
six-1.15.0
sparse-0.12.0
sympy-1.7.1
terminado-0.9.4
testpath-0.4.4
threadpoolctl-2.1.0
torch-1.8.0+rocm4.0.1
torchvision-0.9.0
tornado-6.1
traitlets-5.0.5
typing-extensions-3.7.4.3
urllib3-1.26.4
wcwidth-0.2.5
webencodings-0.5.1
websockets-8.1
widgetsnbextension-3.5.1
yfinance-0.1.55

This is the pip package list with the failed build:

MarkupSafe-1.1.1
Send2Trash-1.5.0
argon2-cffi-20.1.0
async-generator-1.10
attrs-20.3.0
backcall-0.2.0
bleach-3.3.0
certifi-2020.12.5
cffi-1.14.5
chardet-4.0.0
colorcet-2.0.6
cryptography-3.4.7
cvxpy-1.1.11
cycler-0.10.0
decorator-4.4.2
defusedxml-0.7.1
dill-0.3.3
dlx-1.0.4
docplex-2.20.204
ecos-2.0.7.post1
entrypoints-0.3
fastdtw-0.3.4
fastjsonschema-2.15.0
h5py-3.1.0
ibm-quantum-widgets-1.0.3
idna-2.10
inflection-0.5.1
ipykernel-5.5.3
ipython-7.22.0
ipython-genutils-0.2.0
ipyvue-1.5.0
ipyvuetify-1.6.2
ipywidgets-7.6.3
iqx-0.1.17
jedi-0.18.0
jinja2-2.11.3
joblib-1.0.1
json-logging-1.3.0
jsonschema-3.2.0
jupyter-1.0.0
jupyter-client-6.1.13
jupyter-console-6.4.0
jupyter-core-4.7.1
jupyterlab-pygments-0.1.2
jupyterlab-widgets-1.0.0
kaleidoscope-0.0.12
kiwisolver-1.3.1
llvmlite-0.36.0
lxml-4.6.3
matplotlib-3.4.1
mistune-0.8.4
more-itertools-8.7.0
mpmath-1.2.1
multitasking-0.0.9
nbclient-0.5.3
nbconvert-6.0.7
nbformat-5.1.3
nbresuse-0.4.0
nest-asyncio-1.5.1
networkx-2.5.1
notebook-6.3.0
ntlm-auth-1.5.0
numba-0.53.1
numpy-1.20.1
osqp-0.6.2.post0
packaging-20.9
pandas-1.2.3
pandocfilters-1.4.3
param-1.10.1
parso-0.8.2
pexpect-4.8.0
pickleshare-0.7.5
pillow-8.2.0
plotly-4.14.3
ply-3.11
prometheus-client-0.10.0
prompt-toolkit-3.0.18
psutil-5.8.0
ptyprocess-0.7.0
pybind11-2.6.2
pycparser-2.20
pyct-0.4.8
pydot-1.4.2
pygments-2.8.1
pylatexenc-2.10
pyparsing-2.4.7
pyperclip-1.8.2
pyrsistent-0.17.3
pyscf-1.7.3
python-constraint-1.4.0
python-dateutil-2.8.1
pytz-2021.1
pyzmq-22.0.3
qdldl-0.1.5.post0
qiskit-0.25.0
qiskit-aer-0.8.0
qiskit-aqua-0.9.0
qiskit-finance-0.1.0
qiskit-ibmq-provider-0.12.2
qiskit-ignis-0.6.0
qiskit-machine-learning-0.1.0
qiskit-nature-0.1.0
qiskit-optimization-0.1.0
qiskit-terra-0.17.0
qtconsole-5.0.3
qtpy-1.9.0
quandl-3.6.0
requests-2.25.1
requests-ntlm-1.1.0
retrying-1.3.3
retworkx-0.8.0
scikit-learn-0.24.1
scipy-1.6.1
scs-2.1.2
seaborn-0.11.1
setuptools-54.2.0
six-1.15.0
sparse-0.12.0
sympy-1.7.1
terminado-0.9.4
testpath-0.4.4
threadpoolctl-2.1.0
torch-1.8.0+rocm4.0.1
torchvision-0.9.0
tornado-6.1
traitlets-5.0.5
typing-extensions-3.7.4.3
urllib3-1.26.4
wcwidth-0.2.5
webencodings-0.5.1
websockets-8.1
widgetsnbextension-3.5.1
yfinance-0.1.55

This is the nbconvert re-install we do with conda:

## Package Plan ##

  environment location: /opt/conda

  added / updated specs:

    - nbconvert==5.6.1

The following packages will be downloaded:

    package                    |            build

    ---------------------------|-----------------

    nbconvert-5.6.1            |   py38h32f6830_1         488 KB  conda-forge

    numpy-1.20.2               |   py38h9894fe3_0         5.8 MB  conda-forge

    openssl-1.1.1k             |       h7f98852_0         2.1 MB  conda-forge

    ------------------------------------------------------------

                                           Total:         8.4 MB

The following NEW packages will be INSTALLED:

  numpy              conda-forge/linux-64::numpy-1.20.2-py38h9894fe3_0

The following packages will be UPDATED:

  openssl                                 1.1.1j-h7f98852_0 --> 1.1.1k-h7f98852_0

The following packages will be DOWNGRADED:

  nbconvert                            6.0.7-py38h578d9bd_3 --> 5.6.1-py38h32f6830_1

@davidbrochart
Copy link
Member

My concern is if this comes back in the next release (6.2?).

Thanks for reporting, I will investigate before releasing 6.2.

alkino added a commit to BlueBrain/nmodl that referenced this issue Apr 7, 2021
Fix version of jupyter-client, should be removed when the problem with the version 1.6.2 is fixed upstream.

jupyter/jupyter_client#637
nika-qubit pushed a commit to nika-qubit/beam that referenced this issue Apr 7, 2021
1. A new release: jupyter-client 6.1.13 breaks notebooks when getting
   messages.
2. This is caught by screen diff integration tests of interactive beam.
3. The issue is documented at jupyter/jupyter_client#637
   ETA of the fix might be 6.2.x of jupyter-client.
4. Added a timeout=-1 to allow indefinite wait for execution of each
   notebook cell in integration tests. The default value was 1 second.
   Instead of timeout each cell, moved the timeout to each test using pytest mark.
@jph00
Copy link

jph00 commented Apr 8, 2021

@davidbrochart we're seeing the same issue in nbdev. I see Apache Beam also has the issue. Might be worth yanking the 6.1.13 release until there's a fix?

(In the meantime, we're pinning our dep version to work around the issue)

@MSeal
Copy link
Contributor

MSeal commented Apr 8, 2021

@jph00 6.1.13 is yanked from PyPI and marked as broken in conda now

@jph00
Copy link

jph00 commented Apr 8, 2021 via email

alkino added a commit to BlueBrain/nmodl that referenced this issue Apr 8, 2021
We no more need to fix the version because the wrong package (6.1.13) has been
yanked in PyPi

jupyter/jupyter_client#637
alkino added a commit to BlueBrain/nmodl that referenced this issue Apr 9, 2021
* We no more need to fix the version because the wrong package (6.1.13) has been yanked in PyPi

* jupyter/jupyter_client#637
@astafan8
Copy link

I have the same issue with 6.2.0 version :( the project is private, so can't share much, let me know what would be helpful

@davidbrochart
Copy link
Member

@astafan8 thanks for reporting. I ran nbconvert's test suite in jupyter/nbconvert#1549, and except for an unrelated issue, tests are passing: https://travis-ci.org/github/jupyter/nbconvert/builds/766787036
It would be nice if you could share more details.

@vEpiphyte
Copy link

vEpiphyte commented Apr 12, 2021

I am seeing the same issue of test failures with the release of 6.2.0; we had a nightly build failure of our doctests. Here is an example failure (which is visible if you login to circleci) https://app.circleci.com/pipelines/github/vertexproject/synapse/6606/workflows/6f31cd70-94e5-425c-aebc-3c5177b1e18f/jobs/26508

Our notebooks use async code extensively, and reviewing the nbconvert unit tests; they do not appear to have any async code used in them. Its possible that there are unexpected interactions with nest-asyncio and native async applications being executed in notebooks?

@astafan8
Copy link

astafan8 commented Apr 12, 2021

The part that is also puzzling me is that this problem appears for a private project which docs structure and content is similar to that of https://github.com/qcodes/qcodes, and on qcodes latest jupyter-client works without problems.

It would be nice if you could share more details.

of course! but what exactly would be of help?

pip 21.0.1

Python 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32

pip freeze:

jupyter-client==6.1.12  # changing this to 6.1.13 or 6.2.0 results in an exception as in the issue descrption

alabaster==0.7.12
apipkg==1.5
appdirs==1.4.4
argon2-cffi==20.1.0
artifacts-keyring==0.3.1
astroid==2.5.2
async-generator==1.10
atomicwrites==1.4.0
attrs==20.3.0
Babel==2.9.0
backcall==0.2.0
black==20.8b1
bleach==3.3.0
bokeh==2.3.0
broadbean==0.10.0
cached-property==1.5.2
cachetools==4.2.1
certifi==2020.12.5
cffi==1.14.5
cftime==1.4.1
chardet==4.0.0
click==7.1.2
cloudpickle==1.6.0
colorama==0.4.4
colorcet==2.0.6
contextlib2==0.6.0.post1
coverage==5.5
cycler==0.10.0
decorator==4.4.2
defusedxml==0.7.1
docutils==0.16
entrypoints==0.3
et-xmlfile==1.0.1
execnet==1.8.0
google-api-core==1.26.3
google-auth==1.28.0
googleapis-common-protos==1.53.0
h5py==3.2.1
holoviews==1.14.2
hypothesis==6.8.5
idna==2.10
imagesize==1.2.0
importlib-metadata==3.10.0
iniconfig==1.1.1
ipykernel==5.5.3
ipython==7.22.0
ipython-genutils==0.2.0
ipywidgets==7.6.3
isort==5.8.0
jedi==0.18.0
Jinja2==2.11.3
jsonpointer==2.1
jsonschema==3.2.0
junit-xml-2==1.9
jupyter==1.0.0
jupyter-console==6.4.0
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
keyring==23.0.1
kiwisolver==1.3.1
lazy-object-proxy==1.6.0
Markdown==3.3.4
MarkupSafe==1.1.1
matplotlib==3.4.1
mccabe==0.6.1
mistune==0.8.4
mypy==0.812
mypy-extensions==0.4.3
nbclient==0.5.3
nbconvert==5.6.1
nbformat==5.1.3
nbsphinx==0.8.3
nest-asyncio==1.5.1
netCDF4==1.5.6
networkx==2.5.1
notebook==6.3.0
numpy==1.20.2
numpydoc==1.1.0
opencensus==0.7.12
opencensus-context==0.1.2
opencensus-ext-azure==1.0.7
openpyxl==3.0.7
packaging==20.9
pandas==1.2.3
pandocfilters==1.4.3
panel==0.11.1
param==1.10.1
parso==0.8.2
pathspec==0.8.1
pbr==5.5.1
pickleshare==0.7.5
Pillow==8.2.0
pluggy==0.13.1
prometheus-client==0.10.0
prompt-toolkit==3.0.18
protobuf==3.15.7
psutil==5.8.0
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.7.0
pycparser==2.20
pyct==0.4.8
pyflakes==2.3.1
Pygments==2.8.1
pylint==2.7.2
pylint-junit==0.3.2
pyparsing==2.4.7
PyQt5==5.12.3
PyQt5-sip==12.8.1
PyQtWebEngine==5.12.1
pyrsistent==0.17.3
pytest==6.2.3
pytest-cov==2.11.1
pytest-forked==1.3.0
pytest-mock==3.5.1
pytest-timeout==1.4.2
pytest-xdist==2.2.1
python-dateutil==2.8.1
pytz==2021.1
PyVISA==1.11.3
PyVISA-sim==0.4.0
pyviz-comms==2.0.1
pywin32==300
pywin32-ctypes==0.2.0
pywinpty==0.5.7
PyYAML==5.4.1
pyzmq==22.0.3
qcodes==0.24.0
QtAwesome==1.0.2
qtconsole==5.0.3
QtPy==1.9.0
regex==2021.4.4
requests==2.25.1
requirements-parser==0.2.0
rope==0.18.0
rsa==4.7.2
ruamel.yaml==0.17.2
ruamel.yaml.clib==0.2.2
schema==0.7.4
scipy==1.6.2
selenium==3.141.0
Send2Trash==1.5.0
six==1.15.0
snowballstemmer==2.1.0
sortedcontainers==2.3.0
Sphinx==2.4.4
sphinx-jsonschema==1.16.8
sphinx-rtd-theme==0.5.2
sphinxcontrib-apidoc==0.3.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
spyder==3.3.6
spyder-kernels==0.5.2
stringparser==0.5
tabulate==0.8.9
terminado==0.9.4
testpath==0.4.4
toml==0.10.2
tornado==6.1
tqdm==4.60.0
traitlets==5.0.5
typed-ast==1.4.2
typing-extensions==3.7.4.3
urllib3==1.26.4
wcwidth==0.2.5
webencodings==0.5.1
websockets==8.1
widgetsnbextension==3.5.1
wincertstore==0.2
wrapt==1.12.1
xarray==0.17.0
xlrd==2.0.1
zipp==3.4.1

@davidbrochart
Copy link
Member

I pulled 6.2.0 again, there was a API breaking change that I didn't see.
I will fix it and make a 7.0.0 alpha0.

@mriedem
Copy link
Author

mriedem commented Apr 12, 2021

@astafan8 thanks for reporting. I ran nbconvert's test suite in jupyter/nbconvert#1549, and except for an unrelated issue, tests are passing: https://travis-ci.org/github/jupyter/nbconvert/builds/766787036
It would be nice if you could share more details.

One thing I noticed between @astafan8's pip freeze output and ours is we're both using nbconvert 5.6.1 (we're capped on that project until jupyter/nbconvert#1439 is fixed). So running latest nbconvert unit tests with latest jupyter-client might not show the issue but what about nbconvert==5.6.1?

@kirienko
Copy link

Did you try to unpin nbconvert instead of downgrading jupyter-client? It seems to be more beneficial in a long run, since nbconvert-5.6.1 is still aimed to support python-2 and python<3.7.

I had exactly the same issue with nbconvert restricted to <6, but without jupyter_client involved, and unpinning nbconvert resolved the problem.

@mriedem
Copy link
Author

mriedem commented Apr 12, 2021

Did you try to unpin nbconvert instead of downgrading jupyter-client? It seems to be more beneficial in a long run, since nbconvert-5.6.1 is still aimed to support python-2 and python<3.7.

I had exactly the same issue with nbconvert restricted to <6, but without jupyter_client involved, and unpinning nbconvert resolved the problem.

In my case we're pulling in jupyter-client as a transitive dependency. We're capping nbconvert for a separate reason: jupyter/nbconvert#1439

@kirienko
Copy link

Did you try to unpin nbconvert instead of downgrading jupyter-client? It seems to be more beneficial in a long run, since nbconvert-5.6.1 is still aimed to support python-2 and python<3.7.
I had exactly the same issue with nbconvert restricted to <6, but without jupyter_client involved, and unpinning nbconvert resolved the problem.

In my case we're pulling in jupyter-client as a transitive dependency. We're capping nbconvert for a separate reason: jupyter/nbconvert#1439

This performance regression was a good reason to stick to an older version of nbconvert, I fully agree. We thought the same way before it screwed up completely, and we had not another option but unpin it. (I also already faced the dependency issues related to the oldness of 5.6.1 on newer systems.)

Well, I hope this solution you offer will help. However it looks like a workaround for that separate performance issue of nbconvert you mentioned.

@mriedem
Copy link
Author

mriedem commented Apr 13, 2021

One thing I noticed between @astafan8's pip freeze output and ours is we're both using nbconvert 5.6.1 (we're capped on that project until jupyter/nbconvert#1439 is fixed). So running latest nbconvert unit tests with latest jupyter-client might not show the issue but what about nbconvert==5.6.1?

Recreated here jupyter/nbconvert/pull/1550 with jupyter-client 6.2.0 and nbconvert 5.6.1:

https://travis-ci.org/github/jupyter/nbconvert/jobs/766808690#L1370

self = <nbconvert.preprocessors.execute.ExecutePreprocessor object at 0x7f3e7ca1dd30>

msg_id = '238d6f9e-e6ea4495439b51dfd5175f5b_1'

cell = {'cell_type': 'code', 'metadata': {}, 'outputs': [], 'source': 'from __future__ import print_function\nfrom IPython.display import clear_output'}

timeout = 1

    def _poll_for_reply(self, msg_id, cell=None, timeout=None):

        try:

            # check with timeout if kernel is still alive

            msg = self.kc.shell_channel.get_msg(timeout=timeout)

>           if msg['parent_header'].get('msg_id') == msg_id:

E           TypeError: 'coroutine' object is not subscriptable

@davidbrochart
Copy link
Member

@astafan8
Copy link

allow me to report that indeed, using the latest nbconvert as opposed to 5.6.1 removed the issue, and luckily we did not experience any slowdowns in the docs build mentioned in jupyter/nbconvert#1439 .

@mriedem
Copy link
Author

mriedem commented Apr 13, 2021

allow me to report that indeed, using the latest nbconvert as opposed to 5.6.1 removed the issue, and luckily we did not experience any slowdowns in the docs build mentioned in jupyter/nbconvert#1439 .

Cool. Note that the performance slowdown issue there is because we're using object storage with s3fs to back our user notebook pods rather than PVCs (with zero-to-jupyterhub-k8s). The entrypoint import times can get slow with s3fs.

@marius311
Copy link

I could quite follow if the above comments indicate a fix is already in place, but I also had CI fail with this exact error so just wanted to report here in case its helpful.

I also have nbconvert<6 (so 5.6.1) pinned, and I also confirm that pinning jupyter-client==6.1.12 fixes the issue.

Here's the traceback from the CI job: https://github.com/marius311/CMBLensing.jl/runs/2302363389?check_suite_focus=true#step:4:4342

The exact command that fails is jupyter nbconvert --to notebook --execute --inplace --ExecutePreprocessor.timeout=-1 $filename (here)

dpshelio added a commit to UCL-RITS/doctoral-programming-intro that referenced this issue Sep 28, 2021
pinning nbconvert with a particular version that has the templates
jupyter/nbgrader#1373

requires also pin jupyter-client to an earlier version
jupyter/jupyter_client#637
@avivajpeyi
Copy link

avivajpeyi commented Oct 6, 2021

I got the error using jupyter-client==7.0.5 and nbconvert==5.6.1

Full pip freeze: ``` aesara-theano-fallback==0.0.4 alabaster==0.7.12 anyio==3.3.2 argcomplete==1.12.3 argon2-cffi==21.1.0 arviz==0.11.4 astropy==4.0.1 astroquery==0.4.3 attrs==20.3.0 autograd==1.3 Babel==2.9.1 backcall==0.2.0 backports.entry-points-selectable==1.1.0 beautifulsoup4==4.10.0 black==21.9b0 bleach==4.1.0 bokeh==2.4.0 cached-property==1.5.2 cachetools==4.2.4 celerite2==0.2.0 certifi==2021.5.30 cffi==1.14.6 cfgv==3.3.1 cftime==1.5.1 charset-normalizer==2.0.6 click==7.1.2 colorama==0.4.4 corner==2.2.1 cryptography==35.0.0 cycler==0.10.0 Cython==0.29.16 debugpy==1.5.0 decorator==5.1.0 defusedxml==0.7.1 dill==0.3.4 distlib==0.3.3 docutils==0.16 entrypoints==0.3 exoplanet==0.5.1 exoplanet-core==0.1.2 fastprogress==1.0.0 fbpca==1.0 filelock==3.3.0 flake8==3.9.2 future==0.18.2 gitdb==4.0.7 GitPython==3.1.24 greenlet==1.1.2 h5py==3.2.1 html5lib==1.1 identify==2.3.0 idna==3.2 imagesize==1.2.0 importlib-metadata==4.8.1 importlib-resources==5.2.2 iniconfig==1.1.1 ipykernel==6.4.1 ipython==7.28.0 ipython-genutils==0.2.0 ipywidgets==7.6.5 isort==5.9.3 jedi==0.18.0 jeepney==0.7.1 Jinja2==3.0.2 joblib==1.0.1 jsonschema==4.0.1 jupyter==1.0.0 jupyter-book==0.11.3 jupyter-cache==0.4.3 jupyter-client==7.0.5 jupyter-console==6.4.0 jupyter-core==4.8.1 jupyter-server==1.11.1 jupyter-server-mathjax==0.2.3 jupyter-sphinx==0.3.2 jupyterlab-widgets==1.0.2 jupytext==1.10.3 kaleido==0.2.1 keyring==23.2.1 kiwisolver==1.3.2 latexcodec==2.0.1 lightkurve==2.0.11 linkify-it-py==1.0.1 markdown-it-py==0.6.2 MarkupSafe==2.0.1 matplotlib==3.4.3 matplotlib-inline==0.1.3 mccabe==0.6.1 mdit-py-plugins==0.2.6 memoization==0.4.0 mimeparse==0.1.3 mistune==0.8.4 mpi4py==3.0.2 mpmath==1.1.0 mypy-extensions==0.4.3 myst-nb==0.12.3 myst-parser==0.13.7 nbclient==0.5.4 nbconvert==5.6.1 nbdime==3.1.0 nbformat==5.1.3 nest-asyncio==1.5.1 netCDF4==1.5.7 nodeenv==1.6.0 notebook==6.4.4 numpy==1.18.2 oktopus==0.1.2 packaging==21.0 pandas==0.25.3 pandocfilters==1.5.0 parso==0.8.2 pathspec==0.9.0 patsy==0.5.2 pexpect==4.8.0 pickleshare==0.7.5 Pillow==8.3.2 platformdirs==2.4.0 plotly==5.3.1 pluggy==1.0.0 pre-commit==2.15.0 prometheus-client==0.11.0 prompt-toolkit==3.0.20 ptyprocess==0.7.0 py==1.10.0 pybtex==0.24.0 pybtex-docutils==1.0.1 pycodestyle==2.7.0 pycparser==2.20 pydata-sphinx-theme==0.6.3 pyerfa==2.0.0 pyflakes==2.3.1 Pygments==2.10.0 pymc3==3.11.4 pymc3-ext==0.1.0 pyparsing==2.4.7 pyrsistent==0.18.0 pytest==6.2.5 python-dateutil==2.8.2 pytz==2021.3 pyvo==1.1 PyYAML==5.4.1 pyzmq==22.3.0 qtconsole==5.1.1 QtPy==1.11.2 regex==2021.9.30 requests==2.26.0 requests-unixsocket==0.2.0 scikit-learn==1.0 scipy==1.4.1 SecretStorage==3.3.1 semver==2.13.0 Send2Trash==1.8.0 six==1.16.0 smmap==4.0.0 sniffio==1.2.0 snowballstemmer==2.1.0 soupsieve==2.2.1 Sphinx==3.5.4 sphinx-book-theme==0.1.5 sphinx-comments==0.0.3 sphinx-copybutton==0.4.0 sphinx-external-toc==0.2.3 sphinx-jupyterbook-latex==0.4.3 sphinx-multitoc-numbering==0.1.3 sphinx-panels==0.5.2 sphinx-thebe==0.0.10 sphinx-togglebutton==0.2.3 sphinxcontrib-applehelp==1.0.2 sphinxcontrib-bibtex==2.2.1 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 SQLAlchemy==1.4.25 tenacity==8.0.1 terminado==0.12.1 -e git+ssh://git@github.com/dfm/tess-atlas.git@746de72905b4f134e286d699e96200e978c19b60#egg=tess_atlas testbook==0.4.2 testpath==0.5.0 Theano-PyMC==1.1.2 threadpoolctl==3.0.0 toml==0.10.2 tomli==1.2.1 tornado==6.1 tqdm==4.62.3 traitlets==5.1.0 typed-ast==1.4.3 typing-extensions==3.10.0.2 uc-micro-py==1.0.1 uncertainties==3.1.6 urllib3==1.26.7 virtualenv==20.8.1 wcwidth==0.2.5 webencodings==0.5.1 websocket-client==1.2.1 widgetsnbextension==3.5.1 xarray==0.19.0 zipp==3.6.0 ```

I am using myst-nb, which requires nbconvert~=5.6

As @kevin-bates pointed out: pinning jupyter_client==6.1.12 fixes the issue for me

@davidbrochart
Copy link
Member

Maybe worth opening a new issue? This one is for jupyter_client 6.1.13 which was yanked.

elikatsis added a commit to arrikto/kale that referenced this issue Oct 20, 2021
Restrict 'jupyter-client' version to be earlier than 7 to workaround
upstream discrepancies.

Refs jupyter/jupyter_client#637

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
StefanoFioravanzo pushed a commit to arrikto/kale that referenced this issue Oct 20, 2021
Restrict 'jupyter-client' version to be earlier than 7 to workaround
upstream discrepancies.

Refs jupyter/jupyter_client#637

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
Reviewed-by: Stefano Fioravanzo <stefano@arrikto.com>
StefanoFioravanzo pushed a commit to kubeflow-kale/kale that referenced this issue Oct 20, 2021
Restrict 'jupyter-client' version to be earlier than 7 to workaround
upstream discrepancies.

Refs jupyter/jupyter_client#637

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
Reviewed-by: Stefano Fioravanzo <stefano@arrikto.com>
titlethanason added a commit to titlethanason/xqueue_fastapi that referenced this issue Oct 29, 2021
as it has error in autograding with version 7.0.2. The discussion about the error is in jupyter/jupyter_client#637
ldsa-machine pushed a commit to LDSSA/batch5-students that referenced this issue Nov 7, 2021
See more in jupyter/jupyter_client#637. Seems
version jupyter_client 6.1.13 which was yanked is causing issues
@NeilGirdhar
Copy link

This remains broken for me for every jupyter client after 6.1.12.

@davidbrochart
Copy link
Member

Could you elaborate in which context this is happening?

@NeilGirdhar
Copy link

NeilGirdhar commented Nov 14, 2021

@davidbrochart Sure. I run jupyter console in a Python 3.9 virtual environment created by poetry and I can't import anything. If I install jupyter-client 6.1.12, the problem goes away.

@davidbrochart
Copy link
Member

This PR should fix it: jupyter/jupyter_console#244
But I don't have the rights to merge it.

@MSeal
Copy link
Contributor

MSeal commented Nov 14, 2021

@davidbrochart I just merged the PR -- though I need ping someone with release privilages to add me or run a patch release for the repo. I'll check if I can get that added for this week so we can unblock it.

@NeilGirdhar
Copy link

Just wanted to confirm that jupyter/jupyter_console#244 indeed fixed my problem. Thanks, @davidbrochart !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests