Jupytext on JupyterLab 3.0.5 is not automatically syncing notebooks on save

[djakubiec]

I have been successfully using Jupytext with JupyterLab Lab 2 on a remote Ubuntu machine for about a year now. It is great thank you!

I am now trying to get it installed and working with JupyterLab 3 on my macOS machine. All the extensions seem to be loaded fine per the docs, and the configuration shows that I have percent files enabled for my Notebook. But when I save the Notebook it does not generate the .py file.

I was able to get it to sync manually from the command-line (via 'jupytext --sync notebook.ipynb'), but have not been able to get the auto-sync working from within the lab Notebook.

Here are various details. Any suggestions would be greatly appreciated, thanks:

⇒  pip3 list | grep jupyt
jupyter-client          6.1.11
jupyter-core            4.7.0
jupyter-server          1.2.2
jupyterlab              3.0.5
jupyterlab-pygments     0.1.2
jupyterlab-server       2.1.2
jupyterlab-widgets      1.0.0
jupytext                1.9.1

⇒  jupyter nbextension list
Known nbextensions:
  config dir: /Users/dan/.jupyter/nbconfig
    notebook section
      qgrid/extension  enabled
      - Validating: OK
      jupyter-js-widgets/extension  enabled
      - Validating: OK
      jupytext/index  enabled
      - Validating: OK

⇒  jupyter labextension list
JupyterLab v3.0.5
Other labextensions (built into JupyterLab)
   app dir: /Users/dan/Library/Python/3.8/share/jupyter/lab
        @jupyter-widgets/jupyterlab-manager v3.0.0 enabled OK
        jupyterlab-jupytext v1.3.0 enabled OK

⇒  jupyter lab
[I 2021-01-19 22:48:37.582 ServerApp] jupyterlab | extension was successfully linked.
[I 2021-01-19 22:48:37.662 LabApp] JupyterLab extension loaded from /Users/dan/Library/Python/3.8/lib/python/site-packages/jupyterlab
[I 2021-01-19 22:48:37.662 LabApp] JupyterLab application directory is /Users/dan/Library/Python/3.8/share/jupyter/lab
[I 2021-01-19 22:48:37.664 ServerApp] jupyterlab | extension was successfully loaded.
[I 2021-01-19 22:48:37.665 ServerApp] Serving notebooks from local directory: /Users/dan
[I 2021-01-19 22:48:37.665 ServerApp] Jupyter Server 1.2.2 is running at:
[I 2021-01-19 22:48:37.665 ServerApp] http://localhost:8888/lab?token=cbeca734f16afb90988bd682b25424e637d81d3cb06553b7
[I 2021-01-19 22:48:37.665 ServerApp]  or http://127.0.0.1:8888/lab?token=cbeca734f16afb90988bd682b25424e637d81d3cb06553b7
[I 2021-01-19 22:48:37.665 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2021-01-19 22:48:37.678 ServerApp]

    To access the server, open this file in a browser:
        file:///Users/dan/Library/Jupyter/runtime/jpserver-13522-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=cbeca734f16afb90988bd682b25424e637d81d3cb06553b7
     or http://127.0.0.1:8888/lab?token=cbeca734f16afb90988bd682b25424e637d81d3cb06553b7
[I 2021-01-19 22:48:41.740 LabApp] Build is up to date
[W 2021-01-19 22:48:42.318 ServerApp] Notebook fvq/jupyter/marketNeutralEquityStrategy/mnesModels3.ipynb is not trusted
[I 2021-01-19 22:49:05.696 ServerApp] Kernel started: b4f81261-09b1-4c2b-86c4-599951548934
[I 2021-01-19 22:49:17.388 ServerApp] Saving file at /fvq/jupyter/marketNeutralEquityStrategy/mnesModels3.ipynb
[W 2021-01-19 22:49:17.389 ServerApp] Notebook fvq/jupyter/marketNeutralEquityStrategy/mnesModels3.ipynb is not trusted

(Dan note: I "Trusted" the notebook then tried to save again)

[I 2021-01-19 22:49:37.935 ServerApp] Saving file at /fvq/jupyter/marketNeutralEquityStrategy/mnesModels3.ipynb
[I 2021-01-19 22:49:42.285 ServerApp] Saving file at /fvq/jupyter/marketNeutralEquityStrategy/mnesModels3.ipynb

Note the timestamps (.py file never modified prior to manual --sync):

⇒  l mnesModels3.*
-rw-r-----  1 dan  staff   1.8M Jan 19 22:49 mnesModels3.ipynb
-rw-rw----  1 dan  staff   162K Jan 19 22:22 mnesModels3.py

⇒  jupytext --sync mnesModels3.ipynb
[jupytext] Reading mnesModels3.ipynb in format ipynb
[jupytext] Updating 'mnesModels3.py'

⇒  l mnesModels3.*
-rw-r-----  1 dan  staff   1.8M Jan 19 22:49 mnesModels3.ipynb
-rw-rw----  1 dan  staff   162K Jan 19 22:56 mnesModels3.py

What else can I provide?

[mwouts]

Hi @djakubiec , thank you for the detailed report!

The issues seems to be very similar to #713 , i.e. the Jupytext server extension is not loaded when Jupyter starts. Because of that, the contents manager is not substituted by Jupytext's contents manager, and thus you can't use paired files.

May I ask you to try the following?

1. What is the output of jupyter serverextension list? Is Jupytext active there as well? If not, you should activate it and restart Jupyter. Does that solve the issue?
2. Do you get the same issue if you use jupytext==1.8.2? Note that, with that version you won't get the extension for Jupyter Lab 3.0 (i.e. the pair commands) - you will have to install it separately with jupyter labextension install jupyterlab-jupytext@1.3.0.

The above will be mostly useful to find out what is happening. Note that if this really is the same problem as #713, known workarounds are

• to use a new conda environment
• or to set the contents manager manually

[djakubiec]

Thanks for the very speedy response @mwouts.

I checked jupyter serverextension list and it was indeed empty. Sorry it looks like I missed that step from the Install page. I went ahead and ran jupyter serverextension enable jupytext and restarted, but still no luck.

This looks correct now:

⇒  jupyter serverextension list
config dir: /Users/dan/.jupyter
    jupytext  enabled
    - Validating...
      jupytext 1.9.1 OK

But I am not seeing [I 10:28:31.646 LabApp] [Jupytext Server Extension] Changing NotebookApp.contents_manager_class from LargeFileManager to jupytext.TextFileContentsManager.

I further followed the install instructions to create a jupyter_notebook_config.py file (which I did not have) and added the c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager" line... though I did have an existing jupyter_notebook_config.json file. Sadly, none of that helped sorry.

Honestly I am confused about whether I should be using the jupyer_notebook_config.* file(s) or whether I should be making the jupyter_lab_config.* file(s) and doing something there, but FWIW I followed the instructions for now.

Lastly, per your suggestion I also uninstalled jupytext and installed jupytext==1.8.2:

⇒  pip list |grep jupytext
jupytext                1.8.2

I restarted JupyterLab but still no luck.

[djakubiec]

FWIW, I have been studying #713 and it sounds like there is some overlap between our issues. I think my extension is also not loading (based on the missing contents_manager_class log) even though it is now showing up in jupyter serverextension list.

[mwouts]

Thanks @djakubiec for your reports. Thanks for testing jupytext==1.8.2, this way I can exclude a potential regression in 1.9.0.

Is this an option for you to install jupyter and jupytext in a conda environment as documented in the last paragraph in this comment?

That might be easier than trying to use config files (and sorry you can't use the .json file here because of jupyter/nbclassic#41, I'll try to see how to fix that...)

[djakubiec]

I got conda installed and used your suggested environment... all is working great now THANK YOU @mwouts !

I see the advantage of using Anaconda now too, so thanks for that as well.

I suppose there is still some issue getting Jupytext to work with the native install? Let me know if I can help debug anything there... otherwise you are welcome to close this issue if you like.

Thanks again.