tornado 6.0 breaks notebook

[haokanga]

tornado 6.0 is released at 10AM ET, March 1, 2019, which breaks Jupyter Notebook.
https://pypi.org/project/tornado/6.0/#history

Error message

TypeError: 'type' object is not subscriptable

How to reproduce & Error trace

$ python3 -m venv /home/clouduser/virtualenv 
$ source /home/clouduser/virtualenv/bin/activate
$ pip install pandas jupyter jupyterlab protobuf mysqlclient
$ jupyter notebook --no-browser
Traceback (most recent call last):
  File "/home/clouduser/virtualenv/bin/jupyter-notebook", line 7, in <module>
    from notebook.notebookapp import main
  File "/home/clouduser/virtualenv/lib/python3.5/site-packages/notebook/notebookapp.py", line 47, in <module>
    from zmq.eventloop import ioloop
  File "/home/clouduser/virtualenv/lib/python3.5/site-packages/zmq/eventloop/__init__.py", line 3, in <module>
    from zmq.eventloop.ioloop import IOLoop
  File "/home/clouduser/virtualenv/lib/python3.5/site-packages/zmq/eventloop/ioloop.py", line 21, in <module>
    from tornado import ioloop
  File "/home/clouduser/virtualenv/lib/python3.5/site-packages/tornado/ioloop.py", line 45, in <module>
    from tornado.concurrent import (
  File "/home/clouduser/virtualenv/lib/python3.5/site-packages/tornado/concurrent.py", line 175, in <module>
    future: Union["futures.Future[_T]", "Future[_T]"], value: _T
  File "/usr/lib/python3.5/typing.py", line 552, in __getitem__
    dict(self.__dict__), parameters, _root=True)
  File "/usr/lib/python3.5/typing.py", line 512, in __new__
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/usr/lib/python3.5/typing.py", line 512, in <genexpr>
    for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
  File "/usr/lib/python3.5/typing.py", line 190, in __subclasscheck__
    self._eval_type(globalns, localns)
  File "/usr/lib/python3.5/typing.py", line 177, in _eval_type
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
TypeError: 'type' object is not subscriptable

Environments

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial


$python --version
Python 3.5.2

$ pip freeze
attrs==18.2.0
backcall==0.1.0
bleach==3.1.0
decorator==4.3.2
defusedxml==0.5.0
entrypoints==0.3
ipykernel==5.1.0
ipython==7.3.0
ipython-genutils==0.2.0
ipywidgets==7.4.2
jedi==0.13.3
Jinja2==2.10
jsonschema==3.0.1
jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-core==4.4.0
jupyterlab==0.35.4
jupyterlab-server==0.2.0
MarkupSafe==1.1.1
mistune==0.8.4
mysqlclient==1.4.2.post1
nbconvert==5.4.1
nbformat==4.4.0
notebook==5.7.4
numpy==1.16.2
pandas==0.24.1
pandocfilters==1.4.2
parso==0.3.4
pexpect==4.6.0
pickleshare==0.7.5
pkg-resources==0.0.0
prometheus-client==0.6.0
prompt-toolkit==2.0.9
protobuf==3.7.0
ptyprocess==0.6.0
Pygments==2.3.1
pyrsistent==0.14.11
python-dateutil==2.8.0
pytz==2018.9
pyzmq==18.0.0
qtconsole==4.4.3
Send2Trash==1.5.0
six==1.12.0
terminado==0.8.1
testpath==0.4.2
tornado==6.0
traitlets==4.3.2
wcwidth==0.1.7
webencodings==0.5.1
widgetsnbextension==3.4.2

On our side, we have to downgrade tornado to 5.1.1 to address the issue.

pip install tornado==5.1.1

[mpacer]

This issue also breaks on current notebook even with the latest python and tornado 6.0:

#4399

Does your typing issue persist even with a more recent python version?

[mpacer]

Btw, this is a break that is specific it seems to your use of python 3.5.2

it should be fixed once tornado releases their first patch (probably 6.0.1) which should include tornadoweb/tornado@61e1dbf

That commit has the additional message:

The "provisional" typing module in 3.5.2 is kind of broken/incomplete
so we need to use more forward references to avoid confusing it. The
significance of this version in particular is that it was the one
included in ubuntu 16.04.

But don't worry your title still holds… for the way it is handling zmq, tornado 6.0 still breaks notebook.

[ccordoba12]

Pinging @jjhelmus and @mingwandroid about this problem. Please don't update Tornado in Anaconda until this is solved (it seems Spyder is unaffected by this, but still).

[jjhelmus]

Thanks for the heads up @ccordoba12. We will hold back on the tornado release until this is resolved. Additionally, we make look into adding a upper bound on the tornado version to existing notebook conda packages to prevent users from in inadvertently installing an incompatible version.

[btel]

notebook is still broken with tornado 6.0.1. Although the notebook server runs, it fails to connect to the kernel:

[I 12:43:43.849 NotebookApp] The Jupyter Notebook is running at:
[I 12:43:43.849 NotebookApp] http://localhost:8889/?token=e10b38153224b668f74f3839a2fb60db2e7af66d9d3bb96c
[I 12:43:43.849 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:43:43.907 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///run/user/1000/jupyter/nbserver-4991-open.html
    Or copy and paste one of these URLs:
        http://localhost:8889/?token=e10b38153224b668f74f3839a2fb60db2e7af66d9d3bb96c
[I 12:43:48.965 NotebookApp] Creating new notebook in 
[I 12:43:49.771 NotebookApp] Kernel started: cd1d1670-dd78-47a1-b642-d8bedbcdfffd
[I 12:43:50.183 NotebookApp] Adapting to protocol v5.1 for kernel cd1d1670-dd78-47a1-b642-d8bedbcdfffd
/home/bartosz/.pyenv/versions/3.7.2/envs/quantstack-ipywidgets/lib/python3.7/site-packages/notebook/base/zmqhandlers.py:284: RuntimeWarning: coroutine 'WebSocketHandler.get' was never awaited
  super(AuthenticatedZMQStreamHandler, self).get(*args, **kwargs)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[W 12:43:51.208 NotebookApp] Replacing stale connection: cd1d1670-dd78-47a1-b642-d8bedbcdfffd:95982ebe348b4ccc99acbd090554aeb9
[W 12:44:13.262 NotebookApp] Replacing stale connection: cd1d1670-dd78-47a1-b642-d8bedbcdfffd:95982ebe348b4ccc99acbd090554aeb9
[W 12:44:37.266 NotebookApp] Replacing stale connection: cd1d1670-dd78-47a1-b642-d8bedbcdfffd:95982ebe348b4ccc99acbd090554aeb9
[W 12:45:05.303 NotebookApp] Replacing stale connection: cd1d1670-dd78-47a1-b642-d8bedbcdfffd:95982ebe348b4ccc99acbd090554aeb9

Downgrading to tornado 5.1.1 solved the issue.

Update: I installed jupyter through pip on Python 3.7.2

[ccordoba12]

Additionally, we make look into adding a upper bound on the tornado version to existing notebook conda packages to prevent users from in inadvertently installing an incompatible version.

Wise decision!! 👍 Could you add it to ipykernel too? That way Tornado updates wouldn't affect Spyder either.

[rohkamat123]

@bdarnell I think this would be helpful for you to know.

[btel]

Please note that this issue breaks the installation via pip.

[minrk]

This should be fixed by #4449 once that's done, which we can release as 5.7.5

two issues are:

1. our use of deprecated gen.maybe_future which doesn't work with async def coroutines, and
2. self.stream is always None for WebSocketHandlers, which we used to check if the websocket was already closed

Updating these two things should get things going with latest tornado.

As with all cases of latest version compatibility issues, pinning down the breaking package works, e.g.

pip install notebook 'tornado<6'

etc.

Re: conda: please do not add upper bounds to conda packages other than notebook, at least not before confirming that those packages in isolation also have the same issues. I don't think current ipykernel or jupyter-client has these compatibility issues.

[minrk]

Just released 5.7.5 with the fix for tornado 6

[btel]

Thanks @minrk. I confirm that the pip-installed notebook works with tornado 6.0.1 🎉

[zpxz]

I reinstall the jupyter to address the issue.
pip3 install --force-reinstall upgrade jupyter

[laurentperrinet]

pip3 install --force-reinstall upgrade jupyter

you certainly mean:

pip3 install --force-reinstall --upgrade jupyter

[vaka1214]

File "/Library/Frameworks/Python.framework/Versions/3.5/bin/jupyter-notebook", line 5, in
from notebook.notebookapp import main
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/notebook/init.py", line 25, in
from .nbextensions import install_nbextension
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/notebook/nbextensions.py", line 610, in
from .extensions import (
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/notebook/extensions.py", line 8, in
from tornado.log import LogFormatter
ImportError: No module named 'tornado'

can you help me pls?

[brando90]

pip3 install --force-reinstall upgrade jupyter

I tried that but I get that the computer is confused because it asks me to upgrade when other software thinks it has upgraded already thus not allowing an upgrade...weird?

(automl) brandomiranda~ ❯ pip3 install --force-reinstall upgrade jupyter
Collecting upgrade
  Could not find a version that satisfies the requirement upgrade (from versions: )
No matching distribution found for upgrade
You are using pip version 10.0.1, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(automl) brandomiranda~ ❯ pip install --upgrade pip
Requirement already up-to-date: pip in ./miniconda3/envs/automl/lib/python3.7/site-packages (19.3.1)

---

posted in related SO questions:

• https://stackoverflow.com/questions/50914761/jupyter-notebook-cant-find-modules-for-python-3-6
• https://stackoverflow.com/questions/59619442/how-does-one-fix-the-issue-of-not-writable-paths-with-conda

[kevin-bates]

From above ...

pip3 install --force-reinstall upgrade jupyter

you certainly mean:

pip3 install --force-reinstall --upgrade jupyter

[brando90]

Perhaps, but I just copied the command from stack overflow :P

…

Sent from my iPhone

On Jan 8, 2020, at 12:20 PM, Kevin Bates ***@***.***> wrote:  From above ... pip3 install --force-reinstall upgrade jupyter you certainly mean: pip3 install --force-reinstall --upgrade jupyter — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[kevin-bates]

Hmm - not finding a command to upgrade jupyter in the SO posts. However, the point is that you weren't treating upgrade as an option. As a result, pip thinks you're trying to "force reinstall" of a package named "upgrade", when instead the previous comment was meant to upgrade the jupyter package.

Do you see the same behavior when you actually upgrade jupyter via pip3 install --force-reinstall --upgrade jupyter?

[answerquest]

pip3 install --force-reinstall --upgrade jupyter did the job. Thanks! Jupyter notebook is now working at my end with a tornado 6.0.3 installed.