Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make jupyter_packaging standard; add hatchling, editables, pathspec, poetry_core, tomlkit, deprecation; update tomli, flit_core, setuptools, pip, wheel #33866

Closed
mkoeppe opened this issue May 18, 2022 · 40 comments
Closed

Make jupyter_packaging standard; add hatchling, editables, pathspec, poetry_core, tomlkit, deprecation; update tomli, flit_core, setuptools, pip, wheel #33866

mkoeppe opened this issue May 18, 2022 · 40 comments
Labels
c: packages: standard p: major / 3 t: enhancement
Milestone
sage-9.7

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented May 18, 2022

In preparation for an upgrade of jupyter_jsmol and jupyter component packages (#33530), we make the jupyter_packaging package standard. As the latest version of jupyter_packaging now uses hatchling as its build backend, we add it and its dependencies as standard packages and upgrade related packages.

There should be no user-visible change.

CC: @egourgoulhon @dimpase @jhpalmieri @kiwifb @antonio-rojas @tornaria

Component: packages: standard

Author: Matthias Koeppe

Branch: 6607401

Reviewer: Dima Pasechnik, François Bissey, John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/33866
@mkoeppe mkoeppe added this to the sage-9.7 milestone May 18, 2022
@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 27, 2022

Branch: u/mkoeppe/upgrade_jupyter_jsmol__make_jupyter_packaging_standard

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 27, 2022

Author: Matthias Koeppe

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 27, 2022

New commits:

ce9a905build/pkgs/jupyter_jsmol: Update to 2022.1.0
cea777cbuild/pkgs/jupyter_packaging: Make it a normal standard package

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 27, 2022

Commit: cea777c

@mkoeppe

This comment has been minimized.

Sign in to view
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f8c42c7build/pkgs/jupyter_jsmol: Update to 2022.1.0
a843a53build/pkgs/jupyter_packaging: Make it a normal standard package
fbddf49build/pkgs/hatchling: New standard package, build dep of jupyter_client, jsonschema
a3dcb2fbuild/pkgs/editables: New standard package, build dep of hatchling
04b430ebuild/pkgs/tomli: Update to 2.0.1
6d149c9build/pkgs/hatchling/dependencies: Add tomli, editables
6b8c0bebuild/pkgs/flit_core: Update to 3.7.1
bfa7d35build/pkgs/jupyter_packaging: Update to 0.12.2

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Changed commit from cea777c to bfa7d35

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe changed the title Upgrade jupyter_jsmol, make jupyter_packaging standard Make jupyter_packaging standard, add Jun 26, 2022
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

1a63ff6build/pkgs/jupyter_packaging: Make it a normal standard package
a675336build/pkgs/hatchling: New standard package, build dep of jupyter_client, jsonschema
f12287abuild/pkgs/editables: New standard package, build dep of hatchling
f715db8build/pkgs/tomli: Update to 2.0.1
10f216bbuild/pkgs/hatchling/dependencies: Add tomli, editables
4738f8cbuild/pkgs/flit_core: Update to 3.7.1
b82667fbuild/pkgs/jupyter_packaging: Update to 0.12.2

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Changed commit from bfa7d35 to b82667f

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe changed the title Make jupyter_packaging standard, add Make jupyter_packaging standard, add hatchling, editables, update tomli, flit_core Jun 26, 2022
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 26, 2022

comment:8

Deferred the actual update of jupyter_jsmol to a follow-up ticket.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

95e1955build/pkgs/jupyter_packaging/dependencies: Add hatchling
708d846build/pkgs/pathspec: New

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 26, 2022

Changed commit from b82667f to 708d846

@mkoeppe mkoeppe changed the title Make jupyter_packaging standard, add hatchling, editables, update tomli, flit_core Make jupyter_packaging standard; add hatchling, editables, pathspec; update tomli, flit_core Jun 26, 2022
@mkoeppe mkoeppe changed the title Make jupyter_packaging standard; add hatchling, editables, pathspec; update tomli, flit_core Make jupyter_packaging standard; add hatchling, editables, pathspec, poetry_core; update tomli, flit_core, setuptools Jun 26, 2022
@mkoeppe

This comment has been minimized.

Sign in to view
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 27, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

416603fbuild/pkgs/deprecation: New (jupyter_packaging dep)
f5076babuild/pkgs/tomlkit: New (jupyter_packaging dep)
255126abuild/pkgs/setuptools: Update to 62.6.0
5f29fb6build/pkgs/poetry_core: New (tomlkit build dep)
0804b62build/pkgs/poetry_core/patches/5547.diff_bin: New
0c4d0ebbuild/pkgs/hatchling/dependencies: Add pluggy, packaging
3e54474build/pkgs/pip: Update to 22.1.2
7e9722fbuild/pkgs/setuptools_scm: Update to 7.0.2, add typing_extensions as dep
6f72e8cbuild/pkgs/wheel: Update to 0.37.1

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 27, 2022

Changed commit from 708d846 to 6f72e8c

@kiwifb
Copy link
Member

kiwifb commented Jun 28, 2022

comment:23

Nitpick, you did not list a license for hatchling

diff --git a/build/pkgs/hatchling/SPKG.rst b/build/pkgs/hatchling/SPKG.rst
new file mode 100644
index 00000000..23d86bd
--- /dev/null
+++ b/build/pkgs/hatchling/SPKG.rst
@@ -0,0 +1,16 @@
+hatchling: Modern, extensible Python build backend
+==================================================
+
+Description
+-----------
+
+Modern, extensible Python build backend
+
+License
+-------
+
+Upstream Contact
+----------------
+
+https://pypi.org/project/hatchling/
+

It is MIT.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 28, 2022

Changed commit from 3e9d4a8 to 5d8111f

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 28, 2022

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

5d8111fbuild/pkgs/hatchling/SPKG.rst: Add license info

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 28, 2022

comment:25

Thanks!

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 28, 2022

Changed reviewer from Dima Pasechnik to Dima Pasechnik, François Bissey

@jhpalmieri
Copy link
Member

comment:26

Doctest failures, I think because of the pip upgrade:

sage -t --random-seed=40701510628761718232931312764279214628 src/sage/misc/package.py
**********************************************************************
File "src/sage/misc/package.py", line 146, in sage.misc.package.pip_installed_packages
Failed example:
    'scipy' in d  # optional - sage_spkg
Expected:
    True
Got:
    False
**********************************************************************
File "src/sage/misc/package.py", line 148, in sage.misc.package.pip_installed_packages
Failed example:
    d['scipy']  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.7.beta3/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.7.beta3/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.pip_installed_packages[3]>", line 1, in <module>
        d['scipy']  # optional - sage_spkg
    KeyError: 'scipy'

@jhpalmieri
Copy link
Member

comment:27

The issue is that the dictionary pip_installed_packages() now has a key "SciPy" rather than "scipy". I suggested several possible fixes at #33530 comment:68 and #33530 comment:69.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 30, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

6607401src/sage/misc/package.py (pip_installed_packages): Make doctests more flexible

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jun 30, 2022

Changed commit from 5d8111f to 6607401

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 30, 2022

comment:29

Thanks for catching this and your suggestions. I have made similar changes to the doctests. The doctests now accept both formats. I think it's not necessary to track down which of the updated packages is responsible for this change. I wouldn't want to have to tighten version bounds in install-requires.txt for this package just for this doctest.

@jhpalmieri
Copy link
Member

comment:31

Looks good, thank you.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 30, 2022

Changed reviewer from Dima Pasechnik, François Bissey to Dima Pasechnik, François Bissey, John Palmieri

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 30, 2022

comment:32

Thanks!

@vbraun
Copy link
Member

vbraun commented Jul 3, 2022

Changed branch from u/mkoeppe/upgrade_jupyter_jsmol__make_jupyter_packaging_standard to 6607401

@vbraun vbraun closed this as completed in 232d043 Jul 3, 2022
@dimpase
Copy link
Member

dimpase commented Aug 24, 2022

Changed commit from 6607401 to none

@dimpase
Copy link
Member

dimpase commented Aug 24, 2022

comment:34

as it turns out, jupyter_jsmol installation process fetches more js stuff than there is in the tarball:

jjupyter_jsmol-2022.1.0] Building wheels for collected packages: jupyter-jsmol
[jupyter_jsmol-2022.1.0]   WARNING: Ignoring --build-option when building jupyter-jsmol using PEP 517
[jupyter_jsmol-2022.1.0]   Building wheel for jupyter-jsmol (pyproject.toml): started
[jupyter_jsmol-2022.1.0]   Running command Building wheel for jupyter-jsmol (pyproject.toml)
[jupyter_jsmol-2022.1.0]   setup.py entered
[jupyter_jsmol-2022.1.0]   $PATH=/mnt/opt/Sage/sage-dev/build/bin:/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.10/bin:/mnt/opt/Sage/sage-dev/local/bin:/mnt/opt/Sage/sage-dev/build/bin:/mnt/opt/Sage/sage-dev/local/bin:/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.10/bin:/mnt/opt/Sage/sage-dev/build/bin:/mnt/opt/Sage/sage-dev/src/bin:/mnt/opt/Sage/sage-dev/local/bin:/home/dima/.elan/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/14/bin:/home/dima/.local/bin:/home/dima/.local/bin
[jupyter_jsmol-2022.1.0]   <string>:47: DeprecatedWarning: create_cmdclass is deprecated as of 0.8 and will be removed in 1.0. "
[jupyter_jsmol-2022.1.0]   Use `wrap_installers` to handle prebuild steps in cmdclass.
[jupyter_jsmol-2022.1.0]   Use `get_data_files` to handle data files.
[jupyter_jsmol-2022.1.0]   Use `include_package_data=True` and `MANIFEST.in` for package data.
[jupyter_jsmol-2022.1.0] 
[jupyter_jsmol-2022.1.0]   <string>:49: DeprecatedWarning: install_npm is deprecated as of 0.8 and will be removed in 1.0. Use `npm_builder` and `wrap_installers`
[jupyter_jsmol-2022.1.0]   running bdist_wheel
[jupyter_jsmol-2022.1.0]   running jsdeps
[jupyter_jsmol-2022.1.0]   Installing build dependencies with npm.  This may take a while...
[jupyter_jsmol-2022.1.0]   > yarn install
[jupyter_jsmol-2022.1.0]   yarn install v1.22.19
[jupyter_jsmol-2022.1.0]   warning package.json: No license field
[jupyter_jsmol-2022.1.0]   warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[jupyter_jsmol-2022.1.0]   warning jupyter-jsmol@2022.1.0: No license field
[jupyter_jsmol-2022.1.0]   [1/4] Resolving packages...
[jupyter_jsmol-2022.1.0]   [2/4] Fetching packages...
[jupyter_jsmol-2022.1.0]   info There appears to be trouble with your network connection. Retrying...
...

aand it hangs - even tough the network is OK :-(

@dimpase
Copy link
Member

dimpase commented Aug 24, 2022

comment:35

follow-up on #34421

@dimpase dimpase mentioned this issue Aug 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: packages: standard p: major / 3 t: enhancement
Projects
None yet
Milestone
sage-9.7
Development

No branches or pull requests
5 participants
@kiwifb @vbraun @dimpase @jhpalmieri @mkoeppe