mwouts · mwouts · Oct 24, 2023 · Oct 19, 2023 · Oct 19, 2023 · Oct 22, 2023
diff --git a/docs/CHANGELOG.md → CHANGELOG.md b/docs/CHANGELOG.md → CHANGELOG.md
@@ -1,6 +1,14 @@
 Jupytext ChangeLog
 ==================
 
+1.16.0-dev (2023-10-??)
+-------------------
+
+**Changed**
+- Jupytext is now configured with `pyproject.toml` and built with `hatch`. The layout has been reorganised to follow `src-layout` ([#1140](https://github.com/mwouts/jupytext/issues/1140)). This outstanding update was contributed by [Mahendra Paipuri](https://github.com/mahendrapaipuri).
+- The legacy extension for Jupyter Notebook <=6 (the Jupytext Menu) has been removed.
+
+
 1.15.2 (2023-09-16)
 -------------------
 

diff --git a/README.md b/README.md
@@ -1,6 +1,8 @@
 ![](https://github.com/mwouts/jupytext/blob/17aea37c612f33a4e27eeee4b81966f1506920fd/docs/images/logo_large.png?raw=true)
 
-![CI](https://github.com/mwouts/jupytext/actions/workflows/continuous-integration.yml/badge.svg?branch=main)
+<!-- INDEX-START -->
+
+![CI](https://github.com/mwouts/jupytext/actions/workflows/ci.yml/badge.svg?branch=main)
 [![Documentation Status](https://readthedocs.org/projects/jupytext/badge/?version=latest)](https://jupytext.readthedocs.io/en/latest/?badge=latest)
 [![codecov.io](https://codecov.io/github/mwouts/jupytext/coverage.svg?branch=main)](https://codecov.io/gh/mwouts/jupytext/branch/main)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
@@ -12,6 +14,7 @@
 [![Binder:lab](https://img.shields.io/badge/binder-jupyterlab-0172B2.svg)](https://mybinder.org/v2/gh/mwouts/jupytext/main?urlpath=lab/tree/demo/get_started.ipynb)
 [![launch - renku](https://renkulab.io/renku-badge.svg)](https://renkulab.io/projects/best-practices/jupytext/sessions/new?autostart=1)
 [![](https://img.shields.io/badge/YouTube-JupyterCon%202020-red.svg)](https://www.youtube.com/watch?v=SDYdeVfMh48)
+
 # Jupytext
 
 Have you always wished Jupyter notebooks were plain text documents? Wished you could edit them in your favorite IDE? And get clear and meaningful diffs when doing version control? Then, Jupytext may well be the tool you're looking for!

diff --git a/docs/changelog.md b/docs/changelog.md
@@ -0,0 +1,2 @@
+```{include} ../CHANGELOG.md
+```
diff --git a/docs/config.md b/docs/config.md
@@ -41,7 +41,7 @@ The `notebook/` prefix above is matched with the top-most parent folder of the m
 
 ## Global pairing vs individual pairing
 
-Alternatively, notebooks can be paired individually using either the Jupytext commands in Jupyter Lab, the Jupyter Menu in Jupyter Notebook, or the command line interface:
+Alternatively, notebooks can be paired individually using either the Jupytext commands in Jupyter Lab, or the command line interface:
 
 ```bash
 jupytext --set-formats ipynb,py:percent notebook.ipynb

diff --git a/docs/developing.md b/docs/developing.md
@@ -7,13 +7,14 @@ If you want to test a feature that has been integrated in `main` but not deliver
 pip install git+https://github.com/mwouts/jupytext.git
 ```
 
-If you want to test Jupytext in JupyterLab 3 then you will have to build the extension for JupyterLab. To do so, make sure that you have a recent version of `node`, and prefix the command above with `BUILD_JUPYTERLAB_EXTENSION=1`.
+If you want only to build Jupytext core (e.g. not the JupyterLab extension) you can prefix the
+above with `HATCH_BUILD_HOOKS_ENABLE=false`.
 
 Finally, if you want to test a development branch, use
 ```
 pip install git+https://github.com/mwouts/jupytext.git@branch
 ```
-where `branch` is the name of the branch you want to test (again, prefix the command above with `BUILD_JUPYTERLAB_EXTENSION=1` if you want to use Jupytext within JupyterLab 3).
+where `branch` is the name of the branch you want to test.
 
 ## Install and develop Jupytext locally
 
@@ -36,24 +37,19 @@ Tests are executed with `pytest`. You can run them in parallel with for instance
 pytest -n 5
 ```
 
-We also have a `tox.ini` file available if you wish to test your contribution on multiple version of Python before making a PR - just run `tox`.
-
 Build the `jupytext` package and install it with
 ```
-BUILD_JUPYTERLAB_EXTENSION=1 python setup.py sdist bdist_wheel
+pip install -U build wheel
+python -m build
 pip install dist/jupytext-x.x.x-py3-none-any.whl
 ```
 
 or with
 ```
-BUILD_JUPYTERLAB_EXTENSION=1 pip install .
+pip install .
 ```
 
-Finally, note that you can remove `BUILD_JUPYTERLAB_EXTENSION=1` if you don't need the lab extension - the build time will be much shorter.
-
-## Jupytext's extension for Jupyter Notebook
-
-Our extension for Jupyter Notebook adds a Jupytext entry to Jupyter Notebook Menu. The code is found at `jupytext/nbextension/index.js`. Instructions to develop that extension are at `jupytext/nbextension/README.md`.
+Finally, note that you can use `HATCH_BUILD_HOOKS_ENABLE=false` if you don't need the lab extension - the build time will be slightly shorter.
 
 ## Jupytext's extension for JupyterLab
 

diff --git a/docs/faq.md b/docs/faq.md
@@ -10,7 +10,7 @@ The text representation only contains the part of the notebook that you wrote (n
 
 ## How do I use Jupytext?
 
-Open the notebook that you want to version control. _Pair_ the notebook to a script or a Markdown file using either the [Jupytext Commands](install.md#jupytext-commands-in-jupyterlab) in JupyterLab or the [Jupytext Menu](install.md#jupytext-menu-in-jupyter-notebook) in Jupyter Notebook.
+Open the notebook that you want to version control. _Pair_ the notebook to a script or a Markdown file using the [Jupytext Commands](install.md#jupytext-commands-in-jupyterlab) in JupyterLab.
 
 Save the notebook, and you get two copies of the notebook: the original `*.ipynb` file, together with its paired text representation.
 
@@ -38,9 +38,7 @@ The `.ipynb` file contains the full notebook. The paired text file only contains
 
 ## Can I create a notebook from a text file?
 
-Certainly. Open your pre-existing scripts or Markdown files as notebooks with a click in Jupyter Notebook, and with the _Open as Notebook_ menu in JupyterLab.
-
-In Jupyter Notebook you can also create text notebooks with the _New Text Notebook_ menu.
+Certainly. Open your pre-existing scripts or Markdown files as notebooks with the _Open as Notebook_ menu in JupyterLab.
 
 Output cells appear in the browser when you execute the notebook, but they are not written to the disk when you save the notebook.
 
@@ -94,7 +92,7 @@ In this case, a manual action is requested. Remove the paired `.md` or `.py` fil
 
 Jupytext is compatible with JupyterHub (execute `pip install jupytext --user` to install it in user mode) and with Binder (add `jupytext` to the project requirements).
 
-If you use another editor than Jupyter Notebook, Lab or Hub, you probably can't get Jupytext there. However you can still use Jupytext at the command line to manually sync the two representations of the notebook:
+If you use another editor than Jupyter Lab, you probably can't get Jupytext there. However, you can still use Jupytext at the command line to manually sync the two representations of the notebook:
 ```shell
 jupytext --set-formats ipynb,py:light notebook.ipynb   # Pair a notebook to a light script
 jupytext --sync notebook.ipynb                         # Sync the two representations

diff --git a/docs/index.md b/docs/index.md
@@ -1,119 +1,7 @@
-![CI](https://github.com/mwouts/jupytext/actions/workflows/continuous-integration.yml/badge.svg?branch=main)
-[![Documentation Status](https://readthedocs.org/projects/jupytext/badge/?version=latest)](https://jupytext.readthedocs.io/en/latest/?badge=latest)
-[![codecov.io](https://codecov.io/github/mwouts/jupytext/coverage.svg?branch=main)](https://codecov.io/gh/mwouts/jupytext/branch/main)
-[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
-[![GitHub language count](https://img.shields.io/github/languages/count/mwouts/jupytext)](languages.md)
-[![Conda Version](https://img.shields.io/conda/vn/conda-forge/jupytext.svg)](https://anaconda.org/conda-forge/jupytext)
-[![Pypi](https://img.shields.io/pypi/v/jupytext.svg)](https://pypi.python.org/pypi/jupytext)
-[![pyversions](https://img.shields.io/pypi/pyversions/jupytext.svg)](https://pypi.python.org/pypi/jupytext)
-[![Binder:notebook](https://img.shields.io/badge/binder-notebook-0172B2.svg)](https://mybinder.org/v2/gh/mwouts/jupytext/main?filepath=demo)
-[![Binder:lab](https://img.shields.io/badge/binder-jupyterlab-0172B2.svg)](https://mybinder.org/v2/gh/mwouts/jupytext/main?urlpath=lab/tree/demo/get_started.ipynb)
-[![launch - renku](https://renkulab.io/renku-badge.svg)](https://renkulab.io/projects/best-practices/jupytext/sessions/new?autostart=1)
-[![](https://img.shields.io/badge/YouTube-JupyterCon%202020-red.svg)](https://www.youtube.com/watch?v=SDYdeVfMh48)
-
-# Jupytext
-
-Have you always wished Jupyter notebooks were plain text documents? Wished you could edit them in your favorite IDE? And get clear and meaningful diffs when doing version control? Then, Jupytext may well be the tool you're looking for!
-
-## Text Notebooks
-
-A Python notebook encoded in the `py:percent` [format](formats-scripts.md#the-percent-format) has a `.py` extension and looks like this:
-
-```
-# %% [markdown]
-# This is a markdown cell
-
-# %%
-def f(x):
-  return 3*x+1
-```
-
-Only the notebook inputs (and optionally, the metadata) are included. Text notebooks are well suited for version control. You can also edit or refactor them in an IDE - the `.py` notebook above is a regular Python file.
-
-We recommend the `percent` format for notebooks that mostly contain code. The `percent` format is available for Julia, Python, R, and many other [languages](languages.md).
-
-If your notebook is documentation-oriented, a [Markdown-based format](formats-markdown.md) (text notebooks with a `.md` extension) might be more appropriate. Depending on what you plan to do with your notebook, you might prefer the Myst Markdown format, which interoperates very well with Jupyter Book, or Quarto Markdown, or even Pandoc Markdown.
-
-## Installation
-
-Install Jupytext in the Python environment that you use for Jupyter. Use either
-
-    pip install jupytext
-
-or
-
-    conda install jupytext -c conda-forge
-
-Then, restart your Jupyter Lab server, and make sure Jupytext is activated in Jupyter: `.py` and `.md` files have a Notebook icon, and you can open them as Notebooks with a right-click in Jupyter Lab.
-
-![](images/jupyterlab_right_click.png)
-
-## Paired Notebooks
-
-Text notebooks with a `.py` or `.md` extension are well suited for version control. They can be edited or authored conveniently in an IDE. You can open and run them as notebooks in Jupyter Lab with a right-click. However, the notebook outputs are lost when the notebook is closed, as only the notebook inputs are saved in text notebooks.
-
-A convenient alternative to text notebooks are [paired notebooks](paired-notebooks.md). These are a set of two files, say `.ipynb` and `.py`, that contain the same notebook but in different formats.
-
-You can edit the `.py` version of the paired notebook and get the edits back in Jupyter by selecting _reload notebook from disk_. The outputs will be reloaded from the `.ipynb` file if it exists. The `.ipynb` version will be updated or recreated the next time you save the notebook in Jupyter.
-
-To pair a notebook in Jupyter Lab, use the command `Pair Notebook with percent Script` from the Command Palette:
-
-![](images/pair_commands.png)
-
-To pair all the notebooks in a certain directory, create a [configuration file](config.md) with this content:
-
+```{include} ../README.md
+:relative-docs: docs/
+:start-after: <!-- INDEX-START -->
 ```
-# jupytext.toml at the root of your notebook directory
-formats = "ipynb,py:percent"
-```
-
-## Command line
-
-Jupytext is also available at the [command line](using-cli.md). You can
-
-- pair a notebook with `jupytext --set-formats ipynb,py:percent notebook.ipynb`
-- synchronize the paired files with `jupytext --sync notebook.py` (the inputs are loaded from the most recent paired file)
-- convert a notebook in one format to another with `jupytext --to ipynb notebook.py` (use `-o` if you want a specific output file)
-- pipe a notebook to a linter with e.g. `jupytext --pipe black notebook.ipynb`
-
-## Sample use cases
-
-### Notebooks under version control
-
-This is a quick how-to:
-- Open your `.ipynb` notebook in Jupyter and [pair](paired-notebooks.md) it to a `.py` notebook, using either the _pair_ command in Jupyter Lab or a global [configuration file](config.md)
-- Save the notebook - this creates a `.py` notebook
-- Add this `.py` notebook to version control
-
-You might exclude `.ipynb` files from version control (unless you want to see the outputs versioned!). Jupytext will recreate the `.ipynb` files locally when the users open and save the `.py` notebooks.
-
-### Collaborating on notebooks with Git
-
-Collaborating on Jupyter notebooks through Git becomes as easy as collaborating on text files.
-
-Assume that you have your `.py` notebooks under version control (see above). Then,
-- Your collaborator pulls the `.py` notebook
-- They open it _as a notebook_ in Jupyter (right-click in Jupyter Lab)
-- At that stage, the notebook has no outputs. They run the notebook and save it. Outputs are regenerated, and a local `.ipynb` file is created
-- They edit the notebook and push the updated `notebook.py` file. The diff is nothing else than a standard diff on a Python script.
-- You pull the updated `notebook.py` script and refresh your browser. The input cells are updated based on the new content of `notebook.py`. The outputs are reloaded from your local `.ipynb` file. Finally, the kernel variables are untouched, so you have the option to run only the modified cells to get the new outputs.
-
-### Editing or refactoring a notebook in an IDE
-
-Once your notebook is [paired](paired-notebooks.md) with a `.py` file, you can easily edit or refactor the `.py` representation of the notebook in an IDE.
-
-Once you are done editing the `.py` notebook, you will just have to _reload_ the notebook in Jupyter to get the latest edits there.
-
-Note: It is simpler to close the `.ipynb` notebook in Jupyter when you edit the paired `.py` file. There is no obligation to do so; however, if you don't, you should be prepared to read carefully the pop-up messages. If Jupyter tries to save the notebook while the paired `.py` file has also been edited on disk since the last reload, a conflict will be detected, and you will be asked to decide which version of the notebook (in memory or on disk) is the appropriate one.
-
-## More resources
-
-Read more about Jupytext in the [documentation](https://jupytext.readthedocs.io).
-
-If you're new to Jupytext, you may want to start with the [FAQ](faq.md) or with the [Tutorials](tutorials.md).
-
-There is also this short introduction to Jupytext: [![](https://img.shields.io/badge/YouTube-JupyterCon%202020-red.svg)](https://www.youtube.com/watch?v=SDYdeVfMh48).
-
 
 ## Table of Contents
 
@@ -133,5 +21,5 @@ faq.md
 tutorials.md
 contributing.md
 developing.md
-CHANGELOG.md
+changelog.md
 ```
diff --git a/docs/install.md b/docs/install.md
@@ -44,31 +44,32 @@ jupyter labextension enable jupyterlab-jupytext
 jupyter labextension disable jupyterlab-jupytext
 ```
 
-From Jupytext 1.9.0 on, the version of the extension is compatible with JupyterLab 3.x only. If you wish to use Jupytext with JupyterLab 2.x or 1.x, please
+From Jupytext 1.16.0 on, the version of the extension is compatible with JupyterLab 4.x only. If you wish to use Jupytext with JupyterLab 3.x or older, please
 - install the `jupytext` package using `pip` or `conda`
 - and then, install the last version of the `jupyterlab-jupytext` extension that is compatible with your version of JupyterLab, i.e.
 ```
+jupyter labextension install jupyterlab-jupytext@1.3.11  # for JupyterLab 3.x
 jupyter labextension install jupyterlab-jupytext@1.2.2  # for JupyterLab 2.x
 jupyter labextension install jupyterlab-jupytext@1.1.1  # for JupyterLab 1.x
 ```
 
 ## Jupytext menu in Jupyter Notebook
 
-*Note*: as of this writing (version 1.15.1) this section **applies only to
-notebook classic**, i.e. Jupyter Notebook 6.x and below. See [PR
-#1095](https://github.com/mwouts/jupytext/issues/1095) where we discuss how to
-add a Jupytext menu to Jupyter Lab and Notebook 7.x.
+In Jupyter Notebook 7 you can use the same pairing commands as in JupyterLab (see above).
 
-Jupytext includes an extensions for Jupyter Notebook that adds a Jupytext section in the File menu.
+In Jupyter Notebook **classic**, i.e. Jupyter Notebook 6.x and below, Jupytext used to provided an extension that added a Jupytext section in the File menu.
 
 ![](images/jupytext_menu.png)
 
-Here again, the extension should be automatically installed. If need be, you can install and activate it manually with
+That extension is available only for `jupytext<1.16`, and is automatically installed. If need be, you can install and activate it manually with
 ```
 jupyter nbextension install --py jupytext [--user]
 jupyter nbextension enable --py jupytext [--user]
 ```
 
+See also [Issue #1095](https://github.com/mwouts/jupytext/issues/1095) where we discuss how to
+add a Jupytext menu to Jupyter Lab and Notebook 7.x.
+
 ## Jupytext's command line interface
 
 Jupytext provides a `jupytext` command in the terminal that you can use to pair, synchronize or convert notebooks in different formats.

diff --git a/docs/paired-notebooks.md b/docs/paired-notebooks.md
@@ -17,11 +17,7 @@ In JupyterLab, pair your notebook to one or more text formats with the [Jupytext
 
 ![](images/pair_commands.png)
 
-In Jupyter Notebook, use the [Jupytext menu](install.md#jupytext-menu-in-jupyter-notebook):
-
-![](https://raw.githubusercontent.com/mwouts/jupytext/main/jupytext/nbextension/jupytext_menu.png)
-
-These command simply add a `"jupytext": {"formats": "ipynb,md"}` entry to the notebook metadata.
+These commands simply add a `"jupytext": {"formats": "ipynb,md"}` entry to the notebook metadata.
 
 You can also configure the notebook pairing [globally](config.md) for all your notebooks.
 

diff --git a/src/jupytext/version.py b/src/jupytext/version.py
@@ -1,3 +1,3 @@
 """Jupytext's version number"""
 
-__version__ = "1.15.2"
+__version__ = "1.16.0-dev"
diff --git a/tests/test_changelog.py b/tests/test_changelog.py
@@ -34,7 +34,7 @@ def test_replace_issue_numbers_with_links(input, output):
     sys.version_info < (3, 5), reason="'PosixPath' object has no attribute 'read_text'"
 )
 def test_update_changelog():
-    changelog_file = Path(__file__).parent.parent / "docs" / "CHANGELOG.md"
+    changelog_file = Path(__file__).parent.parent / "CHANGELOG.md"
     cur_text = changelog_file.read_text()
     new_text = replace_issue_number_with_links(cur_text)
     if cur_text != new_text: