[RFC 0180] Remove broken packages or unmaintained leaf packages

[Mic92]

Rendered

[dotlambda]

There should be a rule to notify e.g. everyone who touched the package expression in the past before removal.

[Frontear]

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

[Mic92]

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

I am sure that someone wants to become a maintainer of a package like this, once we actually become serious about removing it, because their hardware won't function without it. However we leave the decision what to do in this case up to the person merging the pull request i.e. waiting for a reasonable amount of time depending on the importance package.

[AndersonTorres]

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

There are some of those packages that are very critical. Ideally they should be kept by a team.

[Mic92]

There are some packages with no maintainer yet don't exactly "become outdated" (an example is the intel-microcode package). How are we going to handle those?

There are some of those packages that are very critical. Ideally they should be kept by a team.

I think having this RFC in place, will make actually possible to find instances like this and hopefully improve the maintenance of these packages. I could see the security team for example maintaining microcode package.

[AndersonTorres]

I could see the security team for example maintaining microcode package.

Or the nixos-hardware team (when they become a NixOS team :D )

[Aleksanaa]

I hope the problem infrastructure or something like it can be implemented first, so we can add warnings to end users like "foo is deprecated and is subject to removal because ...", and guide them to give feedback. If they give reasonable reasons, we will cancel or postpone the removal.

We can divide the deprecation of a package into three stages:

1. Throws a warning to end users but still builds on hydra
2. Make a mark similar to insecure. If the user needs to use it, it needs to be manually allowed and it will not be built on hydra.
3. Completely remove related expressions.

Each of these stages can last three to six months, giving everyone plenty of time to react and plenty of reason to remove it.

[Frontear]

Additionally going back to maintainerless packages, I think itd be best to do a broad sweep and find which packages are still orphans and find them a team, especially for critical packages like the microcode package. Once this is done anything left behind would be non essential stuff, which can follow the removal procedure

[8573]

Each of these stages can last three to six months, giving everyone plenty of time to react

Wouldn't anything shorter than six months per stage not be enough time for users of stable channels to notice before the removal process may move on to the next stage?

[AndersonTorres]

About orphaning, we also need to deal with the silent retirement. There are a truckload of maintainers that did not maintain their codes from a long span of time.

I suggested to do a soft deletion of silent retired maintainers at NixOS/nixpkgs#310759.

Regardless the above:

We have the infamous Zero Hydra Failures event. We can do something similar around the months 3 and 9 in order to mark and sweep unmaintained packages.

[Mic92]

I hope the problem infrastructure or something like it can be implemented first, so we can add warnings to end users like "foo is deprecated and is subject to removal because ...", and guide them to give feedback. If they give reasonable reasons, we will cancel or postpone the removal.

We can divide the deprecation of a package into three stages:

1. Throws a warning to end users but still builds on hydra

2. Make a mark similar to insecure. If the user needs to use it, it needs to be manually allowed and it will not be built on hydra.

3. Completely remove related expressions.

Each of these stages can last three to six months, giving everyone plenty of time to react and plenty of reason to remove it.

I don't see really enough benefit in step 2, it just creates more work for us. Removing packages should not be more work than adding packages.

[piegamesde]

Ok, clear. I must have missed "leaf" on my first reading. Still, this is a problem that should be addressed sooner or later.

Agreed, but that's a different issue IMO. It was first tackled in RFC #167, which unfortunately did not end up going very far

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-09-30/53690/1

[AndersonTorres]

I believe situations like these worth some consideration:

NixOS/nixpkgs#345758

[Atemu]

I fail to see how such situations have anything to do with this RFC.

[Naxdy]

I think upstream disappearing is a special case, since all expressions involving the affected packages would no longer build without a binary cache providing the sources (even if nothing else changed), and there is no hope to restoring a functional state.

Since this RFC only mentions broken or unmaintained packages, and imo neither of those labels adequately describes the situation, I think we'd probably need a separate RFC for that one. There are also a lot of special cases to consider, like what if there's already a well-adopted fork out there, what in the case of legal takedowns, should these packages be removed only from unstable or also stable, etc. - things that are not immediately obvious at all. Probably a writeup for another day 😛

[Frontear]

I actually agree with the above for another reason:

If a package disappears upstream, then it propagates to every single revision of nixpkgs that has ever provided that package.

To put it another way, were it not for the binary cache keeping packages, if upstream disappears that derivation essentially becomes one big build failure. Packages like that can potentially pose a problem for users who depended on said package, whether it be now or many releases ago. I do think there's some worth exploring the possible avenues of how to handle it, because there's a lot of nuance involved there, and a simple "mark as broken -> remove" may not be the best decision if we consider the impact radius.

EDIT: I realize I essentially re-stated what the above poster said 😝. I do think this concern has merit, and that this PR doesn't really apply much to it. It would be a good rfc for the future to consider how to deal with sources that have disappeared from the internet. One of nixpkgs' main benefits is being able to go back and use something from long before. Matter of fact I myself used this technique to compile a program which linked to glibc 2.34. There is a pretty decent concern for what should be done when this expectation breaks. Of course its out of nixpkgs' hands and I do not suggest we need to take responsibility in fixing said source, but I do think the discussion can be useful.

[infinisil]

@emilazy @preisi @jopejoe1 @Mic92 You initiated FCP about 3 weeks ago, but there have been some discussions since then. Can you clarify the final decision on whether to accept/reject or have further discussion?

[AndersonTorres]

Reformat and extra suggestion

[Mic92]

@infinisil afaik all shepherds have agreed to this RFC. Now @jopejoe1 just needs to confirm.

[jopejoe1]

@infinisil afaik all shepherds have agreed to this RFC. Now @jopejoe1 just needs to confirm.

Can confirm that all of us shepherds have agreed to this RFC.

[kevincox]

RFCSC: Thanks everyone, this RFC has been accepted!

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-10-28/55095/1