Toolchain Support Policy - EasyBuild 5

[branfosj]

Work has started on EasyBuild 5 and we, the EasyBuild maintainers, are requesting community feedback on some potential changes. If you will be impacted by this proposed change then please leave feedback below.

Proposed Change

For the central EasyBuild repositories, we will support a set of toolchain generations. In year X (example using 2023):

• Accept PRs only for toolchains X-2
  • 2023a/b, 2022a/b, 2021a/b
• Deprecate toolchains X-3
  • 2020a/b
  • Close PRs and issues for these toolchains
• Archive easyconfigs using toolchain version X-4
  • 2019a/b (and older)

Background

The central easyconfigs repository is intended to provide references easyconfigs. For the reference easyconfigs we aim to have only one version of a software being used as a dependency in each toolchain generation. (There are exceptions, but we aim to keep these limited.)

Sites and users can then supplement these easyconfigs via there own collection of easyconfigs. This additional collection is added via the robot search path.

Benefits

• Toolchain generations are year based and this aligns the support with that, rather than EasyBuild versions.
• Focus community effort of the more recent toolchain generations.
• Reduce maintainer workload, and testing time, related to processing easyconfigs from older toolchain generations.

Drawbacks

• There are currently only 4 supported toolchain generations.

Alternatives

Alternative 1

As above, but with each step being 1 year older. So, review (X-3); deprecate and close (X-4); and archive (X-5)

Alternative 2

Base it on the last 6 common toolchains, instead of years. So, review (last 6); deprecate and close (7); and archive (8).

[jfgrimm]

I think I'd prefer to deprecate/archive based on the released toolchains (alternative 2), since that will give a constant number of supported toolchains. Perhaps we should deprecate 7 & 8 though, and archive 9+ (which would put us more in line with when toolchains are archived with the main proposal)?

[boegel]

@branfosj There are currently only 4 supported toolchain generations.

I guess this could be clarified:

There are currently (May 2023) only 4 supported toolchain generations (`2021a`, `2021b`, `2022a`, `2022b`), since the `2023a` common toolchains have not been defined yet (but it's a work in progress).

[boegel]

I think I'd prefer to deprecate/archive based on the released toolchains (alternative 2), since that will give a constant number of supported toolchains. Perhaps we should deprecate 7 & 8 though, and archive 9+ (which would put us more in line with when toolchains are archived with the main proposal)?

I think I agree with @jfgrimm here, since then the moment when we deprecate/archive a toolchain is also clear: as soon as the next common toolchain is defined.
Also, with the year-based approach we would suddenly have to deprecate not 1 but 2 toolchains, and archive 2 toolchains at the start of a new year (but that may slip in February, etc.).

By tying the deprecation/archiving of old toolchain versions to the release of a new common toolchain version, we reduce the load a bit (deprecate 1 + archive 1, rather than 2 each), and we can make that process part of the "release" process of a new common toolchain version (so make those changes in the same EasyBuild release, and bump the minor version).

So, to make this approach more concrete:

a) support the 5 most recent common toolchain versions (currently that would be 2022b, 2022a, 2021b, 2021a, 2020b);
b) deprecate the next 2 common toolchain versions (currently 2020a and 2019b) - and actively close PRs that (only) touch easyconfigs of that generation - we could make the easyconfigs test suite aware of this;
c) archive easyconfigs for older common toolchain versions (currently 2019a, and older);

Maybe the 5 in a) should be 6, I guess that can be up for discussion/vote.

When we define the 2023a common toolchain versions, we would then archive easyconfigs using a 2019b toolchain, and deprecate ones using a 2020b toolchain.

[branfosj]

@branfosj There are currently only 4 supported toolchain generations.

I guess this could be clarified:

There are currently (May 2023) only 4 supported toolchain generations (`2021a`, `2021b`, `2022a`, `2022b`), since the `2023a` common toolchains have not been defined yet (but it's a work in progress).

And

When we define the 2023a common toolchain versions ...

With these, we'll need to agree on what qualifies as having defined common toolchain. I'm guessing currently this is foss/X and intel/X in an EB release.

[hajgato]

I am in favor to deprecate/archive versions when a new one is released, not based on years. So we keep a constant amount of "supported" toolchains.

[boegel]

When we define the 2023a common toolchain versions ...

With these, we'll need to agree on what qualifies as having defined common toolchain. I'm guessing currently this is foss/X and intel/X in an EB release.

To me it means having easyconfigs for foss/2023a and intel/2023a in the develop branch (not necessarily in a release).
As soon as those PRs are merged, we can actively start the deprecated/archiving of the old toolchains (so those changes are included in the next EasyBuild release, along with the new common toolchain version).

We shouldn't do this yet when defining 2023a though (which should happen soon), so the community has some time to provide feedback to this proposal?

[branfosj]

We shouldn't do this yet when defining 2023a though (which should happen soon), so the community has some time to provide feedback to this proposal?

My intention is that we start this later this year. A possible timeline:

• Proposal available for feedback until end of June
• Convert it into a page in the docs
• Probably work one per week, through the old toolchains until we get to whatever we've decided is the latest we'll archive

Currently we have back to 2016a. So if we go with #872 (comment) we'd have 7 to archive (2016a, 2016b, 2017a, 2017b, 2018a, 2018b, and 2019a) and the addition of the 2023a common toolchains would add 2019b to this list.

[signalbox2]

I agree with jfgrimm, as for the less-used software (and we as using bioinformatics packages) there is often no update after the initial burst of effort when the associated paper is published and people want to try it out. So it's not uncommon to want to use quite old easyconfigs as being less effort than updating them to a current toolchain. Myself I try if possible to only use easyconfigs from one toolchain per year to limit sprawl.

[boegel]

@branfosj Maybe we should wait until we switch over from the 5.0.x branch to develop for the development of EasyBuild 5.x, before we start making breaking changes in policy w.r.t. easyconfigs we archive & accept PRs for?

[branfosj]

Thanks everyone for the feedback. I believe we've agreed on the following.

Supported Toolchain Generations Policy

For the central EasyBuild repositories, we will support a set of toolchain generations.

• Accept PRs only for the 6 most recent toolchain generations.
• Deprecate toolchains generations 7 and 8, including closing PRs and issues for these toolchains.
• Archive easyconfigs using toolchain generation 9 (and older).

Notes

• This assumes we continue with two toolchain generations per year.
• A toolchain generation exists when we have an lettered (i.e. 2024a) toolchain in develop.

Background

The central easyconfigs repository is intended to provide references easyconfigs. For the reference easyconfigs we aim to have only one version of a software being used as a dependency in each toolchain generation. (There are exceptions, but we aim to keep these limited.)

Sites and users can then supplement these easyconfigs via there own collection of easyconfigs. This additional collection is added via the robot search path.

Example

As of July 2023 the latest toolchain generation is 2023a.

• Supported: 2022b, 2022a, 2021b, 2021a, 2020b, 2020a
• Deprecated: 2019b and 2019a
• Archived (and unsupported): 2018b and older

[branfosj]

I've PR-ed this: easybuilders/easybuild-docs#200