Support having different WS and HTTP urls

[snowch]

nb2kg does not work when connecting to a host that has different urls for WS and HTTP, for example:

• wss://myhost:8443/gateway/default/jkgws/
• https://myhost:8443/gateway/default/jkg/

This patch allows users to specify the KG_WS_URL.

[parente]

The notebook server has a configuration option for specifying the websocket URL already:

NotebookApp.websocket_url : Unicode
Default: ''

The base URL for websockets, if it differs from the HTTP server (hint: it almost certainly doesn’t).

Should be in the form of an HTTP origin: ws[s]://hostname[:port]

https://jupyter-notebook.readthedocs.io/en/stable/config.html

You should be able to access it via self.settings['websocket_url'] in the handler class where you need it, instead of inventing a new configuration parameter. (See https://github.com/jupyter/notebook/blob/master/notebook/notebookapp.py#L266 for how that setting is populated.)

[snowch]

@parente that setting looks like it only allows you to set the scheme, host and port? The requirement is to be able to set the path.

[parente]

I'm pretty sure it allows the path too. It gets passed to the jinja template renderer here:

https://github.com/jupyter/notebook/blob/74fbc5b5784d5dbadcc66d7a78cedf029d1bb7d9/notebook/base/handlers.py#L401

populates the notebook template here:

https://github.com/jupyter/notebook/blob/5a0ee98441109156a537e3c2a59b6eaf6e8934e1/notebook/templates/notebook.html

and ultimately gets joined to a kernel path here in the JavaScript:

https://github.com/jupyter/notebook/blob/9f578dcec041aa1ad724a27afd2e26b2d79ff099/notebook/static/services/kernels/kernel.js#L461

I think that URL join should work fine even if the ws_url has a path included in it.

Can you give it a shot? If it works, we can send a PR to notebook to update the documentation of the websocket_url traitlet.

[snowch]

I've set the parameter NotebookApp.websocket_url:

cd /home/jovyan/work
. /usr/local/bin/start.sh jupyter notebook --debug \
     --no-browser \
     --port=$NB_PORT \
     --NotebookApp.ip=0.0.0.0 \
     --NotebookApp.websocket_url=$KG_WS_URL \
     --NotebookApp.session_manager_class=nb2kg.managers.SessionManager \
     --NotebookApp.kernel_manager_class=nb2kg.managers.RemoteKernelManager \
     --NotebookApp.kernel_spec_manager_class=nb2kg.managers.RemoteKernelSpecManager 2>&1

However, I only see http calls in the log output, no ws calls.

To enable me to debug this further, is it possible to switch on http/ws trace debugging so the log output shows the actual http/ws requests and responses?

Update: I found out that I can debug the ws calls in the browser in a notebook session.

[snowch]

I've identified some differences in the behavoir ...

This works:

• NotebookApp.websocket_url not set
• KernelGatewayWSClient patch: os.getenv('KG_WS_URL', KG_URL.replace('http', 'ws')),
• Observations:
  • browser variable set: data-ws-url=""
  • debug output in the browser console shows the websocket is trying to talk to my notebook running locally: ws://0.0.0.0:8686/api/kernels/****

This doesn't work:

• NotebookApp.websocket_url is set to my spark cluster
• KernelGatewayWSClient patch: os.getenv('KG_WS_URL', KG_URL.replace('http', 'ws')),
• Observations:
  • browser variable set: data-ws-url="wss%3A//sparkhost%3A8443/gateway/default/jkgws/"
  • debug output in the browser console shows the websocket is trying to talk directly to the remote server: wss://sparkhost:8443/gateway/default/jkgws//api/kernels/*****

 

Suggested alt does not work as expected. Let's go with the env var.

[parente]

@snowch Merging your original since clearly the alternative doesn't do what I think it did. Thanks for experimenting with the command line parameter and your patience.

[snowch]

Thanks, @parente! No worries - it was a really useful adventure into the jupyter source code for me. The code is starting to feel familiar which is nice :)