Install sage_setup with setuptools as a prerequisite for building/installing sagelib #29847

mkoeppe · 2020-06-11T02:46:32Z

sage_setup will be used by the build system of various distribution packages (#29705).

So we make it a regular distribution package itself, with sources in pkgs/sage-setup/ (#31577) and install it with pip.

The actual Python source files are not moved because sage_setup takes part in Sage doctesting (see #31577).

sagelib's setup.py will no longer install sage_setup.

sage-setup will become a build-system requires of sagemath-standard (declared in pyproject.toml)

Follow-up:

sage_setup: Do not fail if sage.env, sage.* are not available #29924: Do not fail if sage.env, sage.* are not available

References:

python - pip ignores dependency_links in setup.py - Stack Overflow
https://stackoverflow.com/questions/12518499/pip-ignores-dependency-links-in-setup-py

dependency_links flag ignored when package exists on PyPI · Issue #987 · pypa/setuptools
pypa/setuptools#987

Suggest alternatives for --process-dependency-links · Issue #6162 · pypa/pip
pypa/pip#6162

PEP 508 -- Dependency specification for Python Software Packages | Python.org
https://www.python.org/dev/peps/pep-0508/#pep440

PEP 440 -- Version Identification and Dependency Specification | Python.org
https://www.python.org/dev/peps/pep-0440/#local-version-segments

Depends on #32046

CC: @kiwifb @jhpalmieri @dimpase

Component: build

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: f84730e

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29847

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-06-11T02:59:08Z

Dependencies: #29702

mkoeppe · 2020-06-12T20:30:49Z

Changed dependencies from #29702 to #29702, #29706, #29785

mkoeppe · 2020-06-13T05:35:02Z

Branch: u/mkoeppe/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup

mkoeppe · 2020-06-14T05:33:28Z

Changed dependencies from #29702, #29706, #29785 to #29702, #29706, #29785, #29855

mkoeppe · 2020-06-14T05:33:28Z

Commit: 4ce81bf

mkoeppe · 2020-06-14T05:33:28Z

Last 10 new commits:

`eed920e`	`src/sage/tests: Move Extension options from src/module_list.py to distutils directives`
`0d25d1d`	`src/sage/structure: Move Extension options from src/module_list.py to distutils directives`
`808f46a`	`src/sage/stats: Move Extension options from src/module_list.py to distutils directives`
`10f7542`	`src/sage/schemes: Move Extension options from src/module_list.py to distutils directives`
`5867c05`	`src/sage/modular/pollack_stevens/dist.pyx: Add missing distutils directives`
`c536daa`	`Remove self-listing in distutils sources directive`
`b1b3787`	`sage.env.cython_aliases: Fix for systems without zlib pc`
`2ba1d7b`	`Merge branch 't/29706/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files' into t/29785/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_3__get_rid_of_uname_specific_`
`bb6cc0c`	`Merge branch 't/29785/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_3__get_rid_of_uname_specific_' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup`
`4ce81bf`	`sage_setup.command.sage_build_cython: Use SAGE_GMP_PREFIX, SAGE_NTL_PREFIX for header dependencies instead of SAGE_INC`

sagetrac-git · 2020-06-14T05:34:06Z

Changed commit from 4ce81bf to 4a9dd3a

sagetrac-git · 2020-06-14T05:34:06Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`041c9e8`	`sage_setup.command.sage_build_cython: Use SAGE_GMP_PREFIX, SAGE_NTL_PREFIX for header dependencies instead of SAGE_INC`
`4a9dd3a`	`Merge branch 't/29855/sagelib_setup_py__fix_dependencies_on_header_files_of_packages_gmp__ntl' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup`

mkoeppe · 2021-07-24T19:11:14Z

Changed dependencies from #31577, #29585, #32073 to #31577, #29585, #32073, #32046

sagetrac-git · 2021-07-24T19:22:36Z

Changed commit from 9aa1f10 to 6b1b222

sagetrac-git · 2021-07-24T19:22:36Z

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

`8c5ee65`	`build/pkgs/pip/install-requires.txt: Update lower bound`
`ed5f9f0`	`Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install`
`6b1b222`	`Merge #32046`

mkoeppe · 2021-07-24T19:23:43Z

comment:87

The branch already contained a part of #32046 but the dependency was not recorded. Fully merged the latest version of #32046. I think we can defer this ticket to 9.5

sagetrac-git · 2021-08-12T21:14:14Z

Changed commit from 6b1b222 to a41b507

sagetrac-git · 2021-08-12T21:14:14Z

Branch pushed to git repo; I updated commit sha1. New commits:

`85d1d27`	`Merge tag '9.4.rc1' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install`
`050f8df`	`build/pkgs/numpy/spkg-install.in: Use sdh_pip_install, sdh_prefix_args`
`a41b507`	`Merge branch 't/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup`

mkoeppe · 2021-08-12T21:14:50Z

Changed dependencies from #31577, #29585, #32073, #32046 to #32046

sagetrac-git · 2021-08-12T22:19:12Z

Changed commit from a41b507 to 478ffcd

sagetrac-git · 2021-08-12T22:19:12Z

Branch pushed to git repo; I updated commit sha1. New commits:

`290fa2f`	`build/pkgs/sagelib/src: Restore lost symlink`
`478ffcd`	`Merge #32046`

sagetrac-git · 2021-08-13T00:31:00Z

Changed commit from 478ffcd to b2368c6

sagetrac-git · 2021-08-13T00:31:00Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b2368c6`	`pkgs/sage-setup/setup.cfg: Add pkgconfig as install_requires`

jhpalmieri · 2021-08-16T23:26:40Z

comment:92

I'm getting a lot of doctest failures:

sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/utils.py  # 15 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/find.py  # 38 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/rdf.py  # 11 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/optional_extension.py  # 12 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/__init__.py  # 5 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/generator.py  # 70 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/clean.py  # 13 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/storage.py  # 97 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/memory.py  # 57 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/cdf.py  # 9 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/cc.py  # 20 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/python.py  # 25 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/instructions.py  # 39 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/rr.py  # 21 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/base.py  # 22 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/autogen/interpreters/specs/element.py  # 11 doctests failed
sage -t --long --warn-long 100.8 --random-seed=0 src/sage_setup/util.py  # 6 doctests failed

The directory local/lib/python3.9/site-packages/sage_setup doesn't have much in it anymore, and I'm guessing that this causes the problems.

sagetrac-git · 2021-09-03T23:26:04Z

Changed commit from b2368c6 to f84730e

sagetrac-git · 2021-09-03T23:26:04Z

Branch pushed to git repo; I updated commit sha1. New commits:

`db8e30d`	`Merge tag '9.5.beta0' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup`
`37f9b0c`	`build/make/Makefile.in: For script packages, make sure that 'sage -f' uninstalls first`
`456d8a9`	`build/make/Makefile.in (sage_setup-clean): Clean the build tree`
`f84730e`	`sage_setup.clean: Do not clean the sage_setup installation in site-packages`

mkoeppe · 2021-09-03T23:27:44Z

comment:95

Thanks for catching this. When configure --enable-editable is not used, the "cleaner" erased sage_setup's files at the end of the sagelib installation. Fixed now, tamed the cleaner.

dimpase · 2021-09-07T17:46:56Z

comment:96

seems to work for me (without --enable-editable).

mkoeppe · 2021-09-07T18:18:35Z

comment:97

Thank you!

vbraun · 2021-09-13T22:22:32Z

Changed branch from u/mkoeppe/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup to f84730e

mkoeppe added this to the sage-9.2 milestone Jun 11, 2020

mkoeppe added c: build labels Jun 11, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Install sage_setup with pip~~ Install sage_setup with pip, move sage_include_directories, cython_aliases from sage.env to sage_setup Jun 12, 2020

This comment has been minimized.

Sign in to view

mkoeppe modified the milestones: sage-9.2, sage-9.3 Jun 14, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Install sage_setup with pip, move sage_include_directories, cython_aliases from sage.env to sage_setup~~ Install sage_setup with pip as a prerequisite for building/installing sagelib Jun 15, 2020

mkoeppe modified the milestones: sage-9.3, sage-9.2 Jun 21, 2020

This comment has been minimized.

Sign in to view

sagetrac-git mannequin added s: needs review and removed s: positive review labels Jul 24, 2021

mkoeppe modified the milestones: sage-9.4, sage-9.5 Jul 24, 2021

mkoeppe added s: needs work and removed s: needs review labels Sep 3, 2021

mkoeppe added s: needs review and removed s: needs work labels Sep 3, 2021

dimpase added s: positive review and removed s: needs review labels Sep 7, 2021

vbraun removed the s: positive review label Sep 13, 2021

vbraun closed this as completed in 859ce9d Sep 13, 2021

Install sage_setup with setuptools as a prerequisite for building/installing sagelib #29847

Install sage_setup with setuptools as a prerequisite for building/installing sagelib #29847

Comments

mkoeppe commented Jun 11, 2020

This comment has been minimized.

mkoeppe commented Jun 11, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Jun 12, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Jun 13, 2020

mkoeppe commented Jun 14, 2020

mkoeppe commented Jun 14, 2020

mkoeppe commented Jun 14, 2020

sagetrac-git mannequin commented Jun 14, 2020

sagetrac-git mannequin commented Jun 14, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Jul 24, 2021

sagetrac-git mannequin commented Jul 24, 2021

sagetrac-git mannequin commented Jul 24, 2021

mkoeppe commented Jul 24, 2021

sagetrac-git mannequin commented Aug 12, 2021

sagetrac-git mannequin commented Aug 12, 2021

mkoeppe commented Aug 12, 2021

sagetrac-git mannequin commented Aug 12, 2021

sagetrac-git mannequin commented Aug 12, 2021

sagetrac-git mannequin commented Aug 13, 2021

sagetrac-git mannequin commented Aug 13, 2021

jhpalmieri commented Aug 16, 2021

sagetrac-git mannequin commented Sep 3, 2021

sagetrac-git mannequin commented Sep 3, 2021

mkoeppe commented Sep 3, 2021

dimpase commented Sep 7, 2021

mkoeppe commented Sep 7, 2021

vbraun commented Sep 13, 2021