Have configure abort if building for 32 bit #11282
Conversation
Per open-mpi#11248, have configure fail if we're building for a 32-bit environment. Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Also explicitly mention that 32-bit builds are no longer supported as of v5.0.0. Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
|
Brought this up on the 10 Jan 2023 webex: no one cared, and no one wanted to be a 32-bit maintainer. Sent this to the devel list for a wider distribution. If no one steps up as maintainer by Fri, 13 Jan 2023, we'll merge this PR (and PR it to v5.0.x). |
|
If it helps justify this, Slurm deprecated 32-bit support four years ago, and we've yet to hear of anyone complaining. Although we do provide a flag (--enable-deprecated) to bypass that check: |
|
@wickberg does this mean, you cannot submit i386 jobs with SLURM anymore. If not, I do not see the connection to this issue? |
|
bot:aws-v2:retest |
1 similar comment
|
bot:aws-v2:retest |
So it is a week after your deadline - any conclusions? |
|
Thanks - I have committed the PMIx version and backported it to the v4.2 branch for inclusion in upcoming release. |
|
So, every architecture here that is 32-bit will then fail once openmpi 5.0.0 is uploaded to Debian unstable:
Since the package has several reverse dependencies, it will make those reverse dependencies unbuildable. Thus, your change would break the following large list of packages on Debian unstable for 32-bit architectures: Not sure whether you really want to do that. |
|
So just to be clear: you are advocating that every software package maintain 32-bit support for all eternity - even if nobody is asking for it? Otherwise, this isn't a discussion of 32-bit or no-32-bit - it is simply a question of when the backward breakage occurs. |
No, I am not advocating for that. I am advocating against breaking software intentionally like what this PR does. There is a difference between »we need to drop this code because it puts a big maintenance burden on us« vs. »I don't want you to use my software on target system XYZ«. |
|
Okay - not trying to be confrontational about this, but the reason we dropped it is the first one. We are a volunteer organization, and we have no volunteers willing to maintain 32-bit support. We know it is broken. So I'm curious to know how you suggest we proceed? We could, I guess, not release OMPI v5, or we could advise Debian not to include it any more. Neither of those seems satisfactory nor helpful to the general community. Given that 32-bit support will not be fixed, how then would you suggest we resolve the dilemma? Letting people try to build it - and fail - seems counterproductive. Not sure what the Debian maintainer would/could do with that situation. Any other suggestions would be welcome. |
|
If you can let us know how it's broken, we're happy to help in fixing it. There are several downstream projects and developers that still work on 32-bit targets, even more exotic ones such as hppa or m68k. Most of these people are volunteers, too. In most cases, developers will just send patches to the corresponding upstream projects if they run into any issues. I know that both Debian and Gentoo developers are doing that regularly for many projects. You don't have to spend any time or effort in maintaining 32-bit support. However, if someone is sending you a patch to fix an issue on a 32-bit target, it would be nice if you could accept it. FWIW, Debian is even working on rebootstrapping 32-bit ARM with 64-bit timestamp support, so I expect there will be enough testing and development on 32-bit targets for the foreseeable future. |
|
We can discuss it - no promises on what we'll decide. All sounds kinda strange, and it would mean putting release timelines outside our own control, so that has to be considered as well. |
|
@glaubitz Can you clarify something for me? I have no doubt that there are valid 32-bit use cases of general computing in a variety of environments. My question is: do these use cases involve Open MPI and/or HPC-class applications? I.e., do you have citeable cases where users are using Open MPI for computation in 32 bit-only environments? I'm not asking to be snarky; I'm asking because HPC applications that are used for academic, scientific, and/or commercial purposes tend to require a lot of compute power, and therefore tends to stay within the last few years of computing hardware. The transition to 64 bit in commodity hardware happened ~20 years ago (per https://en.wikipedia.org/wiki/64-bit_computing); supporting >= 20-year-old hardware is pretty far outside of the envelope of what we typically try to support with Open MPI. We did conduct an informal search to see if we could find any academic, scientific, or commercial users who still use Open MPI in a 32 bit environment, and failed to find any. It certainly wasn't a comprehensive search, and perhaps there are some actual real-world usage of 32 bit MPI out there. If we could find out exactly who those are and what their use cases are, that would be most helpful to the discussion. Thanks! |
|
Figured I would join in on this discussion despite not being that active at this time. 32-bit support is not in any way simple to support even if there are incoming patches. We have components throughout the code base with separate code to handle these archaic architectures. Ideally, we want to clean them all out because modern HPC has long since moved on to 64-bit. Even if there are patches coming in I don't think it is feasible long term to maintain the support. i386, ppc32, arm32, etc are just not worth the effort if 99.999% of usage is on 64-bit. Also, and I know I have said this a lot, if Open MPI 5.0.0 and beyond no longer support 32-bit it doesn't mean there is not an Open MPI version that will run on these architectures. Yes, 32-bit systems will not be able to use new MPI features, but that is to be expected when using something old. Maybe, to keep a working version around around for a bit, it would be worth taking new patches into 4.1.x for 32-bit support. |
|
On Feb 6, 2023, at 4:38 PM, Jeff Squyres ***@***.***> wrote:
@glaubitz Can you clarify something for me? I have no doubt that there are valid 32-bit use cases of general computing in a variety of environments. My question is: do these use cases involve Open MPI and/or HPC-class applications? I.e., do you have citeable cases where users are using Open MPI for computation in 32 bit-only environments?Message ID: ***@***.***>
As outlined earlier in this discussion, the problem are the reverse dependencies.Distributions usually do not ship architecture-specific versions of package but all supported architectures are shipping the same version in their rolling release/unstable distributions.If openmpi did stop working on 32-bit targets, it would mean that the reverse dependencies eventually become unbuildable due to unsatisfied build dependencies because they build-depend on the development files of OpenMPI.
|
|
On Feb 6, 2023, at 5:17 PM, Nathan Hjelm ***@***.***> wrote:
32-bit support is not in any way simple to support even if there are incoming patches. We have components throughout the code base with separate code to handle these archaic architectures. Ideally, we want to clean them all out because modern HPC has long since moved on to 64-bit. Even if there are patches coming in I don't think it is feasible long term to maintain the support. i386, ppc32, arm32, etc are just not worth the effort if 99.999% of usage is on 64-bit.
Out of curiosity: What architecture-specific code are you shipping to support 32-bit PowerPC, for example?
Is there any inline assembly being used?
I’m in no way an MPI expert since my last usage was over 10 years ago, but I would assume the code is mostly high-level C/C++ code so that you don’t have to be aware of the underlying architecture or at least would be able to fall back to generic code, aren’t you?
Also, and I know I have said this a lot, if Open MPI 5.0.0 and beyond no longer support 32-bit it doesn't mean there is not an Open MPI version that will run on these architectures. Yes, 32-bit systems will not be able to use new MPI features but that is to be expected when using something old. Maybe to keep something around for a bit it would be worth taking new patches into 4.1.x for 32-bit support.
As I said before, Linux distributions usually don’t work like that. We cannot ship different package versions for each architecture.
Different package versions are released in different distribution versions. Within one distribution version, a package version is the same across all architectures.
|
|
I believe what this all boils down to is what I said in the beginning - this isn't a question of if we discontinue 32-bit support, but rather when we discontinue it. We can't maintain it forever as that is simply an unrealistic expectation, even if there are outside people willing to contribute patches. As Nathan has indicated, the code just gets too messy. By our own rules, any discontinuation must occur at a major version change. So the question we have to decide is: do we discontinue it at v5? Is there a compelling reason to wait for v6? At the moment, the decision was made to do it for v5, but we'll discuss that again.
Not sure I fully understand this statement. I can download different versions of OMPI for Debian if I want to - been doing that for quite some time, though I confess to mostly using other distros (and so perhaps Debian has limitations I'm unfamiliar with). I do agree this is true for the default version, but that isn't relevant to this discussion.
This seems like a rather odd and somewhat arbitrary constraint. I'm not sure how much weight we should give it in our decision process, but we'll see what people think. |
|
Just for grins, I went to the Debian official package site and found the following OMPI versions available: So one could literally go back to install the old OMPI v2.0.2 release, if so inclined, for any architecture. I therefore don't see the issue with Nathan's proposal - seems like Debian supports it. Agreed, someone interested in 32-bit would need to look for the older versions, but that seems appropriate given the status of 32-bits in general. It does, though, have the secondary impact of constraining such users to the older versions of the dependent packages. Not sure that is an OMPI responsibility, though something worth pondering. |
Understood. But at least a bunch of those packages with build dependencies on Open MPI are:
While I don't know what every single one of the packages is that you cite in your list, I'd be willing to bet that the majority of them are either MPI applications or HPC-related applications for which it's not clear that there's any use case in 32 bit any more. Put differently: do people really have clusters of 32-bit environments where they run HPC applications any more? I guess it boils down to: are there still real-world use cases for HPC/MPI in 32-bit platforms? Or is this just a generalized desire to keep the support matrix for all the packages as large as possible?
Yes. In some of the gnarliest bits of the code, we have some assembly where the standardized libraries do not provide high enough performance. This is what High Performance Computing is all about, after all. 😄 @hjelmn and others can speak in more detail about these parts of the code base than I can. @hjelmn's point above is not incorrect: 32 bit support is not a trivial thing. If none of our users are using it, it's hard to justify the effort to support it.
Someone recently replied on the devel mailing list that they would be willing to support 32 bit -- we'll see where that discussion goes. It is possible that 32 bit will become supported in v5.0.x. |
|
For Debian, we currently support OpenMPI and MPICH as MPI implementations. As @glaubitz points out, the issue is reverse-deps and the fact that we implement only one version of a package in a distribution (release). Disabling 32-bit means either (1) dropping the reverse-dep or MPI support for that package (conditionally for 32-bit only), or (2) switching MPI implementation for that arch. So if 32-bit is removed for OMPI 5.0, we will probably move to MPICH as the default for i386, etc. One possibility is introducing a "new OpenMPI4" package that is just built for 32-bit, ie having 3 implementations (OMPI,MPCICH,OMPI4) as different packages providing libmpi-dev etc. This is technically possible, it depends on |
Those are all packages from different releases. Stretch is Debian 9, Buster is Debian 10, Bullseye is Debian 11. You cannot (easily) mix packages from different releases. And it doesn't help for the reverse dependencies as packages must be built with the packages in the targeting release. |
|
@amckinstry What is Debian's plan for MPICH w.r.t. PMIx? If mpich is built on top of PMIx (this is definitely an option, but I cannot remember if this is mandatory or only optional) and PMIx drops 32bit support as well, does it mean a bumpy road for MPI on 32bit systems? |
|
@ggouaillardet Yes, if pmix goes 64-bit it complicates matters. We do arch-dependent package and configuration deps already (eg ucx is only available on [amd64 arm64 ppc64el] but mpi is built with it on those ). |
|
FWIW: PMIx has never made any effort to have 32-bit support, so I'm rather surprised to hear folks implying that earlier versions supported it. So far as I know, there has been no meaningful change that would have impacted 32 vs 64 bit, so either (a) it never supported 32-bit builds, or (b) it inadvertently broke it. Either way, I'm curious about Debian's plans for the future. Are you saying you must support 32-bit operations for all eternity? So when we switch the norm from today's 64-bit machines to tomorrow's 128-bit machines, you still plan to require everyone to support 32-bit? If not, then again, this is a question of when and not if we drop 32-bit support. So what drives the decision point? |
It's the other way around - ch4 can operate with the old PMI-1, but is limited by it. Of course, for an old 32-bit arch, it won't matter - you are already pretty constrained. |
|
@rhc54 I don't think Debian will support 32-bit "for eternity", and indeed I've had to drop 32-bit support for a bunch of numerical packages that have ceased to care about non-64 bit. MPI is a more complex beast, as is shown by the reverse-dep list @glaubitz posted. There are a bunch of libraries that use MPI that would cause widespread breakage if 32-bit MPI breakage were gone. My current thinking is that we'd move to MPICH (either per-arch or as a default) to retain 32-bit compatibility; |
|
@amckinstry What's your thought on how many users actually run MPI-based apps in 32-bit environments? The list that @glaubitz posted included at least a fair number of MPI applications -- would it really impact any real users if those MPI applications became unavailable in 32-bit environments? Put differently: yes, it looks like a long, scary list of applications that would disappear if 32-bit MPI disappears. But I think the operative question is: given the typical environments where HPC/MPI applications are run, would anyone notice? (that's a genuine question, not a snarky question) |
Not surprising - and I'm not trying to be snarky here either. Just trying to figure out if this whole discussion is based on user demand, or simple inertia. I think the questions from @jsquyres direly address this. Just to be clear, since it is PMIx that is getting dragged over the coals here. We (PMIx) have never made any conscious effort to support 32-bit (as nobody has ever asked for that environment), but we also have no philosophical issue with it. We just never cared since (to our knowledge) nobody uses it there. If someone wants to dive in and make it work, fine - we just don't have anyone interested/willing to do so. I've raised the question of whether OMPI itself even supports 32-bit today, and nobody can say for certain either way. The only reason PMIx is getting raised here is because it is the first compile point that raises the flag - so we cannot get to the rest of the OMPI code to see if it is likewise broken for 32-bit. Ditto for PRRTE (which is also part of OMPI v5). |
|
@jsquyres Its a good question: I doubt there's anyone seriously doing HPC on 32-bit; I'd like to know different. Its mostly a complexity issue: the scary list is not just applications but libraries. Reworking them all to do MPI/non-MPI is significant breakage. So its mostly inertia on avoiding this. |
|
FWIW: I have released PMIx v4.2.3, which is the PMIx being discussed here (it is the minimal level required by OMPI v5): https://github.com/openpmix/openpmix/releases/tag/v4.2.3 I removed the configure "abort if 32-bit" check so PMIx (a) doesn't stand in the way of this conversation, and (b) doesn't get blamed for OMPI losing its "default" status on Debian. Note that the code still doesn't build as 32-bit, so all we've done is move the failure from the "configure" step to the "make" step, which doesn't feel quite right to me...but at least gets me out of the middle of this debate. 😄 If someone wants to work on enabling 32-bit in PMIx, please feel free to do so and submit pull requests. Note that it may be a few months before the next point release, barring someone making it an urgent priority (cookies and other goodies help with such requests). |
|
@rhc54 I might have chosen my words poorly and I apologize if you felt "dragged" into this conversation and/or you thought I was somehow trying to make PMIx the scapegoat here. I was simply trying to exhibit a potential issue that could impact the two major MPI implementations on 32-bit systems, in order to emphasize the important question is (as you already elaborated on that) "when" will (Open) MPI be no more available on upcoming 32-bit systems. I was not willing to discuss the "if" and had zero intent to point finger at "why/who should be blamed" |
|
My apologies, Gilles - I wasn't pointing back to you or anyone. I was only explaining (in a somewhat tongue-in-cheek manner) that I don't want PMIx to "block" OMPI 32-bit support with its configure settings. Barring someone actually resolving the 32-bit warnings Jeff reported, we still won't be able to cleanly compile - which means OMPI can't do so either - but that is a separate question. |
|
FWIW: there is a user on the devel list who is looking into submitting a PR to PMIx for the 32 bit issues: https://www.mail-archive.com/devel@lists.open-mpi.org/msg21447.html |
|
FWIW: I reverted the 32-bit configure abort in the PMIx master branch to make it easier for people to work on those issues: openpmix/openpmix#2960 |
I actually referred to your comment about that (above) in the thread on the devel mailing list 😄 https://www.mail-archive.com/devel@lists.open-mpi.org/msg21457.html |
|
Yeah, but my earlier comment was me removing it from the v4.2 release - not from the master branch. 😜 |
Per #11248, have configure abort if it detects that it is building in a 32-bit environment.
The intent here is to see -- especially after v5.0.0 has been released and gets real-world usage -- if anyone cares about 32-bit builds any more (we suspect that no one does). This PR is trivial to revert if someone cares about 32-bit builds.
If v5.0.x gets "enough" real-world usage and no one cares about 32-bit builds, we can start thinking about removing 32-bit infrastructure from within the Open MPI code base.