Merge pull request #94 from garlick/faq_openmpi

faqs: what versions of OpenMPI work with Flux?

faqs.rst

@@ -115,3 +115,21 @@ The interesting part of the versioning comes from the multi-repo structure. Flux
 A 'flux' meta-package (such as in spack or distro package managers) that would pull in compatible versions of the various sub-packages/repos is also versioned independently of any of its subcomponents. It is a similar situation for the flux-docs repo and the documentation up on readthedocs. Each repo has it's own documentation and that gets tagged and released along with the code, but the high-level "meta" documentation has it's own versioning that is divorced from any particular sub-packages/repos versioning.
 
 .. TODO: we should make a table and put it in the docs too
+
+----------------------------------------
+What versions of OpenMPI work with Flux?
+----------------------------------------
+
+Flux plugins were added to OpenMPI 3.0.0.  Generally, these plugins enable OpenMPI major versions 3 and 4 to work with Flux.  OpenMPI must be configured with the Flux plugins enabled.  Your installed version may be checked with:
+
+.. code-block:: console
+
+ $ ompi_info|grep flux
+                 MCA pmix: flux (MCA v2.1.0, API v2.0.0, Component v4.0.3)
+               MCA schizo: flux (MCA v2.1.0, API v1.0.0, Component v4.0.3)
+
+Unfortunately, `an OpenMPI bug <https://github.com/open-mpi/ompi/issues/6730>`_ broke the Flux plugins in OpenMPI versions 3.0.0-3.0.4, 3.1.0-3.1.4, and 4.0.0-4.0.2.  The `fix <https://github.com/open-mpi/ompi/pull/6764/commits/d4070d5f58f0c65aef89eea5910b202b8402e48b>`_ was backported such that the 3.0.5+, 3.1.5+, and 4.0.2+ series do not experience this issue.
+
+A slightly different `OpenMPI bug <https://github.com/open-mpi/ompi/pull/8380>`_ caused segfaults of MPI in ``MPI_Finalize`` when UCX PML was used.  `The fix <https://github.com/open-mpi/ompi/pull/8380>`_ was backported to 4.0.6 and 4.1.1.  If you are using UCX PML in OpenMPI, we recommend using 4.0.6+ or 4.1.1+.
+
+For the upcoming 5.0 release, the OpenMPI project dropped the Flux plugins and abstraction layer that contains them.  There are two solutions being discussed, neither of which is implemented yet:  `Running PRRTE inside a Flux allocation <https://github.com/flux-framework/flux-core/issues/3539>`_ and `Implementing a PMIx job shell plugin <https://github.com/flux-framework/flux-core/issues/3536>`_.

spell.en.pws

@@ -477,3 +477,4 @@ dmesg
 eventlog
 eventlogs
 nodelist
+backported