GSoC 2024 Submission ‐ Swiss Army Knife Builder CLI

Swiss Army Knife Builder CLI

Author: Ronan Coutinho @cronan03
Mentor: Frédéric Collonval (@fcollonval)
Organization: Numfocus

Goal

The goals of this project are:

• to extract that tooling as a new separate package to ease maintenance (for core and extension developers)
• update the existing package to use the new one
• make it configurable to be reused for other applications.

What I did

1. Successfully created a CLI with the processes develop, build and watch mentioned above.
2. Created extensive unit tests using Pytest to ensure the processes run efficiently on any OS.
3. Reduced external dependencies by bringing jlpm and jupyterlab.semver to the package.
4. Pre released the package. It can be found on Pypi here https://pypi.org/project/jupyter-builder/
5. Initiated a solution to the issue https://github.com/jupyterlab/jupyterlab/issues/13456

What's left to do

1. We should bring @jupyterlab/builder within this package and make it generic. For now the code lives there: https://github.com/jupyterlab/jupyterlab/tree/main/builder
2. https://github.com/jupyterlab/jupyter-builder/blob/fffb100fc57ecb147bface4441f91bfd0cb6ff9a/jupyter_builder/federated_extensions.py#L296 which is responsible for checking version overlap has been temporarily ignored to make the build feature work.
3. Update JupyterLab core to use this new package

Merged PRs

1. https://github.com/jupyterlab/jupyter-builder/pull/11
   • This PR focuses on extracting the develop feature which is responsible for installing the Jupyter extension JS assets in dev mode.
   • Considering the size of labextension.py, only features essential to Jupyter builder were added.
   • Each of the features will inherit from the class BaseExtensionApp present here
   • The federated_extensions.py sets up and executes commands to build, develop and waatch a JupyterLab extension. It resolves paths, constructs the appropriate command-line arguments, and executes the build process using subprocess.check_call. Optional parameters allow for customization of the build process, including logging, development mode, and source map generation.
2. https://github.com/jupyterlab/jupyter-builder/pull/13
   • This PR focuses on extracting the build feature which is responsible for creating the Javascript assets which will be consumed by the Jupyter App.
   • It will always result in the creation of a file static/style.js in <extension_folder>/myextension/labextension.
   • Tests have been crafted using Pytest to check for the existence of files mentioned above on running the build command.
3. https://github.com/jupyterlab/jupyter-builder/pull/18
   • The watch feature on running will rebuild the JS assets on being triggered. This happens on changing contents in <extension_folder>/src/index.ts
   • To test this feature we deliberately make a change in index.ts triggering watch. This replaces old JS assets with new ones having different hash values in the file names. We create 2 vectors of filenames before and after triggering watch which will tell us if it actually worked.
4. https://github.com/jupyterlab/jupyter-builder/pull/20
   • To reduce external dependencies, we added jlpm to this package.
   • It existed here with the entrypoint.
5. https://github.com/jupyterlab/jupyter-builder/pull/22
   • Documented the working of the Jupyter builder along with installation guide.

Challenges and Learning

1. One of the main challenges was starting this project from scratch with no pre existing code to rely on. I thank my mentor @fcollonval for creating the skeleton in https://github.com/jupyterlab/jupyter-builder/pull/2 which gave me a base to work on.
2. Selecting relevant features for Jupyter builder from labextension.py was really tough without a thorough understanding on which functions Jupyter builder would actually rely on.
3. Creating tests for the watch feature was tricky as I had to carefully adjust sleep times to make sure the function was running before a change in <extension_folder>/src/index.ts was made. Otherwise the change happened before watch ran and never triggered it.