[WIP] improve config checks for Boost.MPI and Boost.Python

[lexming]

This is needed to fix the test in easybuilders/easybuild-easyconfigs#11145

There are three fixes:

• fixes sanity checks for Boost >= v1.69.0: as MT libraries have an additional x64 suffix
• explicitly disables MPI if boost_mpi is False: there is a possible case of having usempi=True and boost_mpi=False and in the current easyblock Boost MPI is built
• remove setting usempi=True in line 142 if boost_mpi=True, the MPI sanity check in line 119 will never allow a situation where that is needed

Update:
Disabling MPI is not as simple as adding --without-mpi to b2. The build command only allows for the set of features to be either enabled or disabled, not a mixture of both. Therefore, I revamped the configuration step by adding additional checks on configopts, boost_mpi and the toolchain. These changes not only explicitly disable MPI if boost_mpi is not True, but also check settings provided by the easyconfig in configopts.

I also added similar checks for Python. From now on, adding --without-libraries=python to configopts won't be necessary in the easyconfigs of Boost, it will be automatically added if Python is not in the dependency list.

Behaviour of existing easyconfigs should be unaffected by this PR.

[migueldiascosta]

I was testing this with all the existing Boost easyconfigs and it is failing with Boost-1.70.0-iimpi-2019a.eb, boost_mpi is not being built although it's supposed to... it seems that the using mpi line in user-config.jam is being ignored by bjam (?)

But except for Boost-1.70.0-iimpi-2019a.eb, it works with all the other easyconfigs, including Boost-1.70.0-gompi-2019a.eb...

Update: this particular issue was unrelated to this PR and a local problem, it disappeared after reinstalling the intel/2019a toolchain.

[migueldiascosta]

also, this seems to break all the Boost.Python easyconfigs - bjam doesn't seem to like the explicit --without-mpi when --with-python is used (?)

[lexming]

@migueldiascosta thanks for the tests! I'll check those issues ASAP

[lexming]

@migueldiascosta Enabling and disabling features in the build step is indeed not trivial and breaks Boost.Python. However, that led into a rabbit hole and I added way more changes than originally planned. Apart from the fix of MT libs in the sanity check, the other changes add better handling of errors in the provided configuration options related to Boost.MPI and Boost.Python. That should fix all previous issues. I'll add test reports asap.

[lexming]

The following tests on Boost with and without Python are all positive:

• Boost-1.55.0-foss-2016a-Python-2.7.11.eb: https://gist.github.com/lexming/7c16fc807c408d3408b2b5155efbeced
• Boost-1.65.1-foss-2017b.eb: https://gist.github.com/lexming/ca796333dbf897a105785ae93556bf87
• Boost-1.65.1-foss-2017b-Python-2.7.14.eb: https://gist.github.com/lexming/3daa9f82032a1698a1dbc065ee70bc0a
• Boost-1.67.0-intel-2018b.eb: https://gist.github.com/lexming/9a6e3557e09306d2e7fe384708c2eccb
• Boost-1.68.0-intel-2018b-Python-3.6.6.eb: https://gist.github.com/lexming/ad6b92a520db39bcdf62568726832409
• Boost-1.71.0-gompi-2019b.eb: https://gist.github.com/lexming/1d6474dc32c6ef9892887bd041198bc4
• Boost-1.71.0-iimpi-2019b.eb: https://gist.github.com/lexming/4662e79e3ff741a7ff4eb921e16ce11e
• Boost-1.72.0-gompi-2020a.eb: https://gist.github.com/lexming/ac818ae59f09395adad1ca3882f93ef2
• Boost-1.72.0-iimpi-2020a.eb: https://gist.github.com/lexming/81fecb08066898e6ec4e60c909dac3ed

[lexming]

The following tests on Boost.Python are positive:

• Boost.Python-1.64.0-gompi-2019a.eb: https://gist.github.com/lexming/9c0dec46abfcbb2616bcdcaa0087e7dd
• Boost.Python-1.67.0-foss-2018b-Python-2.7.15.eb: https://gist.github.com/lexming/bf1eb375751cba010f936590af87fb41
• Boost.Python-1.67.0-foss-2018b-Python-3.6.6.eb: https://gist.github.com/lexming/8d1a49e261b7482fc9884f4c014a362a
• Boost.Python-1.71.0-gompi-2019b.eb: https://gist.github.com/lexming/7fe57278575b37d2f162b37a041088c2
• Boost.Python-1.71.0-iimpi-2019b.eb: https://gist.github.com/lexming/d98097c35ca79d729dd9df3a684f3f2d
• Boost.Python-1.72.0-gompi-2020a.eb: https://gist.github.com/lexming/f81e2111e82056bcc3d78cde62c4c270
• Boost.Python-1.72.0-iimpi-2020a.eb: https://gist.github.com/lexming/d7230470d452e7c8bfb33c7406645082

[lexming]

Tests on a few hypothetical cases:

1. Setting boost_mpi = False and usempi = True proceeds and libboost_mpi is not installed

• Example EC: https://gist.github.com/lexming/f2813bf1e251031c741a878fd080645a
• Log: https://gist.github.com/lexming/5e10403586567867ca461540dba302cd

2. Contradictory settings provided by user in configopts with boost_mpi will trigger an error

• Example EC: https://gist.github.com/lexming/6f3773a259bd6990020ae9ea78243e5d
• Log: https://gist.github.com/lexming/12fd112af8d70867ffce1bd28d45eda9

3. Python bindings will not be installed unless Python is listed as dependency or user explicitly enables those in configopts

• Example EC: https://gist.github.com/lexming/0483c45b7a557ad82514802cb458469b
• Log: https://gist.github.com/lexming/6dd787be9fe27eee85e91f3ddbed1128

[migueldiascosta]

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

• SUCCESS Boost.Python-1.64.0-gompi-2019a.eb
• SUCCESS Boost.Python-1.64.0-gompic-2019a.eb
• SUCCESS Boost.Python-1.65.1-intel-2017b-Python-2.7.14.eb
• SUCCESS Boost.Python-1.66.0-foss-2018a-Python-3.6.4.eb
• SUCCESS Boost.Python-1.66.0-intel-2018a-Python-2.7.14.eb
• SUCCESS Boost.Python-1.66.0-intel-2018a-Python-3.6.4.eb
• SUCCESS Boost.Python-1.67.0-foss-2018b-Python-2.7.15.eb
• SUCCESS Boost.Python-1.67.0-foss-2018b-Python-3.6.6.eb
• SUCCESS Boost.Python-1.67.0-fosscuda-2018b-Python-2.7.15.eb
• SUCCESS Boost.Python-1.67.0-intel-2018b-Python-2.7.15.eb
• SUCCESS Boost.Python-1.67.0-intel-2018b-Python-3.6.6.eb
• SUCCESS Boost.Python-1.70.0-gompi-2019a.eb
• SUCCESS Boost.Python-1.70.0-iimpi-2019a.eb
• SUCCESS Boost.Python-1.71.0-gompi-2019b.eb
• SUCCESS Boost.Python-1.71.0-iimpi-2019b.eb
• SUCCESS Boost.Python-1.71.0-iimpic-2019b.eb
• SUCCESS Boost.Python-1.72.0-gompi-2020a.eb
• SUCCESS Boost.Python-1.72.0-iimpi-2020a.eb
• SUCCESS Boost-1.71.0-gompic-2019b.eb
• SUCCESS Boost.Python-1.71.0-gompic-2019b.eb

Build succeeded for 20 out of 20 (19 easyconfigs in total)
sms - Linux centos linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor (zen), Python 2.7.5
See https://gist.github.com/749fb0c4fc2811791e716c1c94291326 for a full test report.

[migueldiascosta]

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

• SUCCESS Boost-1.55.0.eb
• FAIL (build issue) Boost-1.58.0-GCC-4.9.2-serial.eb (partial log available at https://gist.github.com/cbbe240fcf7d9042146a6a5171cb49d0)
• SUCCESS Boost-1.63.0-foss-2018a-Python-2.7.14.eb
• SUCCESS Boost-1.64.0-gompi-2019a.eb
• SUCCESS Boost-1.64.0-gompic-2019a.eb
• SUCCESS Boost-1.66.0-foss-2018a.eb
• SUCCESS Boost-1.66.0-foss-2018a-Python-2.7.14.eb
• SUCCESS Boost-1.66.0-foss-2018a-Python-3.6.4.eb
• SUCCESS Boost-1.66.0-GCCcore-6.4.0-no_mpi.eb
• SUCCESS Boost-1.66.0-intel-2018.01.eb
• SUCCESS Boost-1.66.0-intel-2018.01-Python-3.6.3.eb
• SUCCESS Boost-1.66.0-intel-2018a.eb
• SUCCESS Boost-1.66.0-intel-2018a-Python-2.7.14.eb
• SUCCESS Boost-1.66.0-intel-2018a-Python-3.6.4.eb
• SUCCESS Boost-1.67.0-foss-2018a-Python-2.7.14.eb
• SUCCESS Boost-1.67.0-foss-2018b.eb
• SUCCESS Boost-1.67.0-fosscuda-2018b.eb
• SUCCESS Boost-1.67.0-intel-2018a.eb
• SUCCESS Boost-1.67.0-intel-2018b.eb
• SUCCESS Boost-1.68.0-foss-2018b-Python-2.7.15.eb
• SUCCESS Boost-1.68.0-foss-2018b-Python-3.6.6.eb
• SUCCESS Boost-1.68.0-intel-2018b-Python-2.7.15.eb
• SUCCESS Boost-1.68.0-intel-2018b-Python-3.6.6.eb
• SUCCESS Boost-1.69.0-intel-2019.01.eb
• SUCCESS Boost-1.70.0-gompi-2019a.eb
• SUCCESS Boost-1.70.0-iimpi-2019a.eb
• SUCCESS Boost-1.70.0-iimpic-2019a.eb
• SUCCESS Boost-1.71.0-gompi-2019b.eb
• SUCCESS Boost-1.71.0-gompic-2019b.eb
• SUCCESS Boost-1.71.0-iimpi-2019b.eb
• SUCCESS Boost-1.71.0-iimpic-2019b.eb
• SUCCESS Boost-1.72.0-GCCcore-9.3.0-no_mpi.eb
• SUCCESS Boost-1.72.0-gompi-2020a.eb
• SUCCESS Boost-1.72.0-iimpi-2020a.eb

Build succeeded for 33 out of 34 (34 easyconfigs in total)
sms - Linux centos linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor (zen), Python 2.7.5
See https://gist.github.com/5f919f319346c1cb5c4d79af1b09851b for a full test report.

[migueldiascosta]

@lexming this breaks Boost-1.58.0-GCC-4.9.2-serial.eb (and I suppose any easyconfig with a --with-libraries in configopts for which Boost.MPI and/or Boost.Python are explicitly disabled by this easyblock via --without-libraries?)