From ac8307a7112b0c6d763c9470213ad95fcb722ce2 Mon Sep 17 00:00:00 2001 From: Kenneth Hoste Date: Wed, 12 Apr 2023 21:21:28 +0200 Subject: [PATCH 1/5] add page with overview of changes in EasyBuild v5.0 (WIP) --- docs/easybuild-v5/overview-of-changes.md | 93 ++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/easybuild-v5/overview-of-changes.md diff --git a/docs/easybuild-v5/overview-of-changes.md b/docs/easybuild-v5/overview-of-changes.md new file mode 100644 index 000000000..62077eaa7 --- /dev/null +++ b/docs/easybuild-v5/overview-of-changes.md @@ -0,0 +1,93 @@ +# Overview of changes in EasyBuild version 5.0 {: #overview } + +This page provides a concise overview of the most prominent changes in EasyBuild version 5.0, +which can be categorized as: + +* [Significant enhancements][significant_enhancements] +* [Backward-incompatible changes][backwards_incompatible] +* [Deprecated functionality][deprecated] + +For in-depth details on a particular change, see the pull requests that are linked from each of the subsections below. + +!!! warning + + EasyBuild 5.0 is currently still under development, via the `5.0.x` branches in the EasyBuild GitHub repositories. + + We intend to update this page regularly as the planned changes are being implemented. + + +--- + +## Significant enhancements in EasyBuild v5.0 {: #significant_enhancements } + +Various significant enhancements are included in EasyBuild v5.0, including: + +* *(no significant enhancements implemented so far in the `5.0.x` branches)* + +--- + +## Backwards-incompatible changes in EasyBuild v4.0 {: #backwards_incompatible } + +A number of *backwards-incompatible* changes are being made in EasyBuild v5.0: + +* [Support for Python 2.7 is removed -- Python 3.6+ is required][py36] +* [Deprecated EasyBuild bootstrap script is removed][bootstrap_script] +* [Experimental support for the .yeb easyconfig format is removed][yeb] + +--- + +### Support for Python 2.7 is removed -- Python 3.6+ is required {: #py36 } + +EasyBuild 5.0 requires Python >= 3.6 to run. + +Running EasyBuild with Python 2.7 or a Python 3 version older than Python 3.6 is no longer supported. + +Trying to run EasyBuild with a Python version that is too old will result in an error: +``` +ERROR: No compatible 'python' command found via $PATH (EasyBuild requires Python 3.6+) +``` + +Python 2.7 has been [end-of-life since 1 Jan 2020](https://www.python.org/doc/sunset-python-2), +and dropping compatibility with Python 2.7 and Python 3.5 enabled some significant code cleanup +(see [easybuild-framework PR #4229](https://github.com/easybuilders/easybuild-framework/pull/4229)). + +The [results of the 6th EasyBuild User Survey (2022)](https://easybuild.io/user_survey) show that the impact of +this breaking change on the EasyBuild community should be very limited, since: + +* Only ~13% of survey participants were still running EasyBuild on top of Python 2.7; +* No survey participants reported using Python 3.5; +* Over 85% of survey participants reported using Python 3.6, or a more recent version of Python 3; +* Only 3 out of 118 survey participants (~2.5%) reported that dropping support for running EasyBuild + on top of Python 2 would be *problematic* for them; + +Along with actively removing code that was only required to retain compatibility with Python 2.7 or 3.5, +the `easybuild.tools.py2vs3` module that was introduced to facilitate supporting both Python 2.7 and Python 3 +has been deprecated ([see also below][py2vs3]). + +--- + +### Deprecated EasyBuild bootstrap script is removed {: #bootstrap_script } + +[easybuild-framework PR #4233](https://github.com/easybuilders/easybuild-framework/pull/4233) + +--- + +### Experimental support for the .yeb easyconfig format is removed {: #yeb } + +[easybuild-framework PR #4237](https://github.com/easybuilders/easybuild-framework/pull/4237) + +--- + +## Deprecated functionality in EasyBuild v5.0 {: #deprecated } + +Some functionality is being deprecated in EasyBuild v5.0, and will no longer be supported in EasyBuild v6.0: + +* [`easybuild.tools.py2vs3` module][py2vs3] + +If you trigger any deprecated functionality, a warning message will be printed. + +--- + +### `easybuild.tools.py2vs3` module {: #py2vs3 } + +[easybuild-framework PR #4229](https://github.com/easybuilders/easybuild-framework/pull/4229) From 0b7c61c88c9a459cfb7b02dd1fd0986bc2488c74 Mon Sep 17 00:00:00 2001 From: Simon Branford Date: Sun, 14 May 2023 11:27:14 +0100 Subject: [PATCH 2/5] planned and proposed changes, and py2vs3 info --- docs/easybuild-v5/overview-of-changes.md | 71 ++++++++++++++++++++++-- 1 file changed, 66 insertions(+), 5 deletions(-) diff --git a/docs/easybuild-v5/overview-of-changes.md b/docs/easybuild-v5/overview-of-changes.md index 62077eaa7..a634ec200 100644 --- a/docs/easybuild-v5/overview-of-changes.md +++ b/docs/easybuild-v5/overview-of-changes.md @@ -1,5 +1,11 @@ # Overview of changes in EasyBuild version 5.0 {: #overview } +!!! warning + EasyBuild 5.0 is currently still under development, via the `5.0.x` branches in the EasyBuild GitHub repositories. + + We intend to update this page regularly as the [planned changes][eb5_plans] are being implemented and when there + are [proposed changes][eb5_proposals] where we are requesting community feedback. + This page provides a concise overview of the most prominent changes in EasyBuild version 5.0, which can be categorized as: @@ -9,12 +15,32 @@ which can be categorized as: For in-depth details on a particular change, see the pull requests that are linked from each of the subsections below. -!!! warning +--- - EasyBuild 5.0 is currently still under development, via the `5.0.x` branches in the EasyBuild GitHub repositories. +## Planned and proposed changes for EasyBuild v5.0 {: #eb5_changes } + +At the [EasyBuild User Meetin](https://easybuild.io/eum23), Simon Branford set out the +[roadmap for EasyBuild v5.0](https://easybuild.io/eum23/#easybuild5). + +### Proposed changes for EasyBuild v5.0 {: #eb5_proposals } - We intend to update this page regularly as the planned changes are being implemented. +There are several proposed changes where the EasyBuild maintainers are seeking community feedback. If you wish to provide +feedback then please comment in the GitHub issue for the proposal. +* Minimum supported Lmod Version: _proposal forthcoming_ +* Toolchain Support Policy: _proposal forthcoming_ + +### Planned changes for EasyBuild v5.0 {: #eb5_plans } + +* [enable `--trace` by default](https://github.com/easybuilders/easybuild-framework/pull/4250) + +!!! note + This list is the major planned changes. It is not intended to be a complete list of all changes that are + planned for EasyBuild v5.0. + +### Tracking development of EasyBuild v5.0 + +* [GitHub Project board for EasyBuild v5.0](https://github.com/orgs/easybuilders/projects/18) --- @@ -43,6 +69,7 @@ EasyBuild 5.0 requires Python >= 3.6 to run. Running EasyBuild with Python 2.7 or a Python 3 version older than Python 3.6 is no longer supported. Trying to run EasyBuild with a Python version that is too old will result in an error: + ``` ERROR: No compatible 'python' command found via $PATH (EasyBuild requires Python 3.6+) ``` @@ -68,13 +95,16 @@ has been deprecated ([see also below][py2vs3]). ### Deprecated EasyBuild bootstrap script is removed {: #bootstrap_script } -[easybuild-framework PR #4233](https://github.com/easybuilders/easybuild-framework/pull/4233) +The EasyBuild bootstrap script has been removed (see [easybuild-framework PR #4233](https://github.com/easybuilders/easybuild-framework/pull/4233)). +Please see the [installation page][installation] for the suggested methods for installing EasyBuild. --- ### Experimental support for the .yeb easyconfig format is removed {: #yeb } -[easybuild-framework PR #4237](https://github.com/easybuilders/easybuild-framework/pull/4237) +Support for the experimental `.yeb` easyconfig format has been removed (see [easybuild-framework PR #4237](https://github.com/easybuilders/easybuild-framework/pull/4237)). +This format allowed easyconfigs to be specified in YAML. However, there has been no recent development of this +format and little suggestion that anyone was using it. --- @@ -91,3 +121,34 @@ If you trigger any deprecated functionality, a warning message will be printed. ### `easybuild.tools.py2vs3` module {: #py2vs3 } [easybuild-framework PR #4229](https://github.com/easybuilders/easybuild-framework/pull/4229) + +The following table lists the changes required to replace imports from the the `py2vs3` module. + +| `from easybuild.tools.py2vs3 import ...` | Replacement | +|--|--| +| `ascii_letters` | `from string import ascii_letters` | +| `ascii_lowercase` | `from string import ascii_lowercase` | +| `build_opener` | `from urllib.request import build_opener` | +| `ConfigParser` | `from configparser import ConfigParser` | +| `configparser` | `import configparser` | +| `create_base_metaclass` | `from easybuild.base.wrapper import create_base_metaclass` | +| `extract_method_name` | No import required. Replace `extract_method_name(method)` with `'_'.join(method.__code__.co_names)` | +| `HTMLParser` | `from html.parser import HTMLParser` | +| `HTTPError` | `from urllib.request import HTTPError` | +| `HTTPSHandler` | `from urllib.request import HTTPSHandler` | +| `json_loads` | `from json import loads` and rename `json_loads` to `loads` | +| `Mapping` | `from collections.abc import Mapping` | +| `mk_wrapper_baseclass` | `from easybuild.base.wrapper import mk_wrapper_baseclass` | +| `OrderedDict` | `from collections import OrderedDict` | +| `raise_with_traceback` | No import required. Replace `raise_with_traceback(exception, message, tb)` with `raise exception(message).with_traceback(tb)` | +| `reload` | `from importlib import reload` | +| `Request` | `from urllib.request import Request` | +| `string_type` | No import required. Use `str` directly. | +| `StringIO` | `from io import StringIO` | +| `std_urllib` | `import urllib.request as std_urllib` | +| `subprocess_popen_text` | `from easybuild.tools.run import subprocess_popen_text` | +| `subprocess_terminate` | `from easybuild.tools.run import subprocess_terminate` | +| `urlencode` | `from urllib.parse import urlencode` | +| `URLError` | `from urllib.request import URLError` | +| `urlopen` | `from urllib.request import urlopen` | + From cc2e9da8e9784a83fef9d81fff6d094b28b008ca Mon Sep 17 00:00:00 2001 From: Simon Branford Date: Sun, 14 May 2023 12:20:47 +0100 Subject: [PATCH 3/5] fix ci failure --- docs/easybuild-v5/overview-of-changes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/easybuild-v5/overview-of-changes.md b/docs/easybuild-v5/overview-of-changes.md index a634ec200..0967968ee 100644 --- a/docs/easybuild-v5/overview-of-changes.md +++ b/docs/easybuild-v5/overview-of-changes.md @@ -11,7 +11,7 @@ which can be categorized as: * [Significant enhancements][significant_enhancements] * [Backward-incompatible changes][backwards_incompatible] -* [Deprecated functionality][deprecated] +* [Deprecated functionality][deprecated_v5] For in-depth details on a particular change, see the pull requests that are linked from each of the subsections below. @@ -70,7 +70,7 @@ Running EasyBuild with Python 2.7 or a Python 3 version older than Python 3.6 is Trying to run EasyBuild with a Python version that is too old will result in an error: -``` +```log ERROR: No compatible 'python' command found via $PATH (EasyBuild requires Python 3.6+) ``` @@ -108,7 +108,7 @@ format and little suggestion that anyone was using it. --- -## Deprecated functionality in EasyBuild v5.0 {: #deprecated } +## Deprecated functionality in EasyBuild v5.0 {: #deprecated_v5 } Some functionality is being deprecated in EasyBuild v5.0, and will no longer be supported in EasyBuild v6.0: From caa4e30d7fddb16d7741dfb083f37f5a90b62737 Mon Sep 17 00:00:00 2001 From: Simon Branford <4967+branfosj@users.noreply.github.com> Date: Mon, 15 May 2023 09:09:56 +0100 Subject: [PATCH 4/5] deprecation warning on .yeb page --- docs/writing-yeb-easyconfig-files.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/docs/writing-yeb-easyconfig-files.md b/docs/writing-yeb-easyconfig-files.md index 752773ab0..6b299cf67 100644 --- a/docs/writing-yeb-easyconfig-files.md +++ b/docs/writing-yeb-easyconfig-files.md @@ -1,12 +1,7 @@ -# Writing easyconfig files in YAML syntax (`.yeb` format) **[IN DEVELOPMENT]** {: #easyconfig_yeb_format } +# Writing easyconfig files in YAML syntax (`.yeb` format) **Deprecated** {: #easyconfig_yeb_format } -!!! note - Because support for easyconfig files in YAML syntax (a.k.a. `.yeb` files) is still *in development*, - using them currently requires enabling the use of experimental features (`--experimental`), - see also [Experimental features][experimental_features]. - - An up-to-date overview of current progress on support for `.yeb` easyconfigs is available at - . +!!! warning + Use of `.yeb` easyconfigs is deprecated and [will be removed in EasyBuild v5.0][yeb]. Useful links: From 719b194ebc46ee8d61edb70076936895fcdff506 Mon Sep 17 00:00:00 2001 From: Kenneth Hoste Date: Wed, 24 May 2023 17:37:22 +0200 Subject: [PATCH 5/5] fix typo (Meetin -> Meeting) --- docs/easybuild-v5/overview-of-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/easybuild-v5/overview-of-changes.md b/docs/easybuild-v5/overview-of-changes.md index 0967968ee..267f6d7c1 100644 --- a/docs/easybuild-v5/overview-of-changes.md +++ b/docs/easybuild-v5/overview-of-changes.md @@ -19,7 +19,7 @@ For in-depth details on a particular change, see the pull requests that are link ## Planned and proposed changes for EasyBuild v5.0 {: #eb5_changes } -At the [EasyBuild User Meetin](https://easybuild.io/eum23), Simon Branford set out the +At the [EasyBuild User Meeting](https://easybuild.io/eum23), Simon Branford set out the [roadmap for EasyBuild v5.0](https://easybuild.io/eum23/#easybuild5). ### Proposed changes for EasyBuild v5.0 {: #eb5_proposals }