New EasyBlock for SuperLU #860

besserox · 2016-03-08T18:09:10Z

No description provided.

hpcugentbot · 2016-03-08T18:09:11Z

Automatic reply from Jenkins: Can I test this?

boegel · 2016-03-08T18:17:43Z

Jenkins: ok to test

boegel · 2016-03-08T18:18:48Z

easybuild/easyblocks/s/superlu.py

+        self.cfg['separate_build_dir'] = True
+
+        if self.cfg['build_shared_libs']:
+            self.cfg['configopts'] += "-DBUILD_SHARED_LIBS=ON "


when updating self.cfg, use self.cfg.update:

self.cfg.update('configopts', '-DBUILD_SHARED_LIBS=ON')

note that you don't need to worry about including a trailing space, update will do that for you

hpcugentbot · 2016-03-08T18:19:51Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1770/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-08T18:20:47Z

easybuild/easyblocks/s/superlu.py

+        # Set the BLAS library to use
+        # For this, use the BLA_VENDOR option from the FindBLAS module of CMake
+        # cf https://cmake.org/cmake/help/latest/module/FindBLAS.html
+        if get_software_root('IMKL'):


use imkl (the official software name), although it matters little in practice

you're also missing a version check here, since the value you set is specific to imkl 10.x?

besserox · 2016-03-08T21:17:00Z

@boegel @fgeorgatos Thanks for reviewing. Some comments below:

About the version check for Intel MKL. The possible values of BLA_VENDOR for Intel are:
- Intel10_32 -> Intel mkl v10 32 bit
- Intel10_64lp -> Intel mkl v10 64 bit,lp thread model, lp64 model
- Intel10_64lp_seq -> Intel mkl v10 64 bit,sequential code, lp64 model
- Intel -> older versions of mkl 32 and 64 bit
I have added a check and I pick either Intel10_64lp or Intel
About the BLAS fallback:
- Generic is looking for a library named blas. It picks /usr/lib/libblas.so on our cluster.
- SuperLU also provide a BLAS implementation not necessarily fast (as they say) that we could fallback to if we set enable_blaslib=ON
- We can also set BLA_VENDOR="All". Then FindBLAS will try all and use the first one it finds. On our cluster this one find ATLAS, ie /usr/lib/libf77blas.so.3gf;/usr/lib/libatlas.so.3gf even the Intel toolchain is loaded. I changed the fallback to this one.
- Or do you prefer to fail?
BTW, you can look at <CMake install>/share/cmake-*/Modules/FindBLAS.cmake to see how it searches for the BLAS libraries.
About PIC, what is the default value? Do we need to add toolchainopts = {'pic': True} in the easyconfig?
About the licence, no problem. I have added "Ghent University" in the copyright line. Is that fine like this, or do you want Contributor Agreement line too?

hpcugentbot · 2016-03-08T21:26:27Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1771/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-09T08:07:39Z

easybuild/easyblocks/s/superlu.py

+        else:
+            # Fallback: try everything, pick the first one
+            # CMake should fail if none is found
+            self.cfg.update('configopts', '-DBLA_VENDOR="All"')


I prefer failing here.

The EB policy is to use the BLAS available in the toolchain. If it's not available, we shouldn't let SuperLU pick another random BLAS library it may find...

boegel · 2016-03-09T08:11:06Z

@besserox: pic is not enabled by default, so if you need it, you should enable it in the easyconfig;

license looks fine now, but the important thing is that you're agreeing with the contributor agreement (if not, you should close this PR)

…to superlu

hpcugentbot · 2016-03-09T09:21:13Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1773/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

besserox · 2016-03-09T09:21:28Z

@boegel Some more comments:

So now, it fails if none of imlk, ACML, ATLAS or OpenBLAS. Is there any other to be supported?

We could also have an easyblock option like force_bla_vendor so this can be overridden in the easyconfig. What do you think?

Is there a better way to find which BLAS is used in the toolchain? For example something like toolchain.options['blas_vendor']
About Intel MKL version
- Intel10_32 (Intel mkl v10 32 bit) -> Does EasyBuild support 32 bit build? How to check?
- Intel10_64lp (Intel mkl v10 64 bit,lp thread model, lp64 model) -> The one we use for IMKL >= 10
- Intel10_64lp_seq (Intel mkl v10 64 bit,sequential code, lp64 model) -> NO, SuperLU requires multithread
- Intel (older versions of mkl 32 and 64 bit) -> The one we use for IMKL < 10
I assume/hope that Intel10_64lp also work for Intel MKL 11.x. SuperLU is compiling fine and all tests run fine. For the FindBLAS.cmake script, it looks like this version difference is only use to choose with library name to look for.
About pic, I have added this to the easyconfig files in PR Add easyconfigs for SuperLU library easybuild-easyconfigs#2665
About the license, I agree with the contributor agreement

boegel · 2016-03-09T09:45:11Z

@besserox:

you've covered the main BLAS libraries with those, I'd say that's OK for now (imkl and OpenBLAS are by far the most prevalent);
for now, we don't have something better than get_software_root to check for the BLAS library being used; I guess it would be useful to have the equivalent to self.toolchain.mpi_family which is already there... We do have self.toolchain.definition(), which returns a dictionary with toolchain components. If there's a BLAS in the toolchain, you could use self.toolchain.definition()['BLAS'], which would provide a list of names for the BLAS library (should be only one entry for BLAS)
So, I guess you can consider using something like:

toolchain_blas = self.toolchain.definition().get('BLAS', ['NO_BLAS_IN_TOOLCHAIN'])[0]
if toolchain_blas == 'OpenBLAS':
    ...
elif ...

boegel · 2016-03-09T09:46:54Z

@besserox: we do support installing the 32-bit version of imkl, but it's not easy to detect that currently... With the exception of an ancient toolchain, all imkl's we use in toolchains are 64-bit.

boegel · 2016-03-09T19:54:44Z

easybuild/easyblocks/s/superlu.py

+        elif toolchain_blas == 'OpenBLAS':
+            # Unfortunately, OpenBLAS is not recognized by FindBLAS from CMake,
+            # we have to specify the OpenBLAS library manually
+            self.cfg.update('configopts', '-DBLAS_LIBRARIES="${EBROOTOPENBLAS}/lib/libopenblas.a;-pthread"')


use get_software_libdir('OpenBLAS') instead of ${EBROOTOPENBLAS}/lib? That's clearer (full path specified), and it deals with the possibility that the library is actually in lib64 (for whatever reason).

@boegel BTW, the usage of get_software_libdir() break the --extended-dry-run functionality. In the sense that, because of this, it reports

!!! !!! WARNING: ignoring error 'NoneType' object has no attribute 'startswith' !!!

instead of the actual configure command line.

Any way to fix this? (not here, but in a general way)

…to superlu

besserox · 2016-03-10T13:08:16Z

@boegel I fixed your remarks

hpcugentbot · 2016-03-10T13:19:26Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1808/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-03-14T17:46:16Z

@besserox informed me that this is tied to #863 (SuiteSparse easyblock update), which needs a bit of attention, so I'm retargeting this until after the EB v2.7.0 release

boegel · 2016-05-10T20:19:46Z

@besserox the window for EasyBuild v2.8.0 is closing this week, any updates on this?

…to superlu

besserox · 2016-05-12T20:18:39Z

@boegel I have not re-tested, but it should be ready to go

boegel · 2016-05-13T17:45:52Z

looks great, testing the easyconfigs in easybuilders/easybuild-easyconfigs#2665...

boegel · 2016-05-13T20:13:22Z

Successful test reports in easybuilders/easybuild-easyconfigs#2665, so good to go.

Thanks @besserox!

Add SuperLU easybock

973a441

boegel added this to the v2.7.0 milestone Mar 8, 2016

boegel reviewed Mar 8, 2016
View reviewed changes

besserox added 3 commits March 8, 2016 22:04

Fix comments

801fcdf

Try all BLAS libraries as a fallback

3d81341

Check for Intel MKL version to choose the right BLA_VENDOR

5c1b2c9

boegel reviewed Mar 9, 2016
View reviewed changes

besserox added 3 commits March 9, 2016 10:04

Fail if no BLAS library is found among the toolchain

0d0f5af

Better comments

b085274

Merge branch 'develop' of github.com:hpcugent/easybuild-easyblocks in…

98a3390

…to superlu

besserox changed the title ~~[WIP] New EasyBlock for SuperLU~~ New EasyBlock for SuperLU Mar 9, 2016

boegel reviewed Mar 9, 2016
View reviewed changes

boegel modified the milestones: v2.x, v2.7.0 Mar 9, 2016

besserox added 2 commits March 10, 2016 13:55

Many fixes

07c4fe9

Merge branch 'develop' of github.com:hpcugent/easybuild-easyblocks in…

d26dfd8

…to superlu

boegel modified the milestones: v2.7.0, v2.x Mar 11, 2016

besserox mentioned this pull request Mar 11, 2016

Add easyconfigs for SuperLU library easybuilders/easybuild-easyconfigs#2665

Merged

boegel modified the milestones: v2.8.0, v2.7.0 Mar 14, 2016

Merge branch 'develop' of github.com:hpcugent/easybuild-easyblocks in…

f9e31b4

…to superlu

boegel merged commit 9d94880 into easybuilders:develop May 13, 2016

besserox deleted the superlu branch May 24, 2016 08:30

ocaisa mentioned this pull request May 25, 2022

enhance SuperLU easyblock to support building on top of FlexiBLAS and be compatible with SuperLU v5.3 #2722

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New EasyBlock for SuperLU #860

New EasyBlock for SuperLU #860

besserox commented Mar 8, 2016

hpcugentbot commented Mar 8, 2016

boegel commented Mar 8, 2016

boegel Mar 8, 2016

hpcugentbot commented Mar 8, 2016

boegel Mar 8, 2016

besserox commented Mar 8, 2016

hpcugentbot commented Mar 8, 2016

boegel Mar 9, 2016

boegel commented Mar 9, 2016

hpcugentbot commented Mar 9, 2016

besserox commented Mar 9, 2016

boegel commented Mar 9, 2016

boegel commented Mar 9, 2016

boegel Mar 9, 2016

besserox Mar 10, 2016

besserox commented Mar 10, 2016

hpcugentbot commented Mar 10, 2016

boegel commented Mar 14, 2016

boegel commented May 10, 2016

besserox commented May 12, 2016

boegel commented May 13, 2016

boegel commented May 13, 2016

New EasyBlock for SuperLU #860

New EasyBlock for SuperLU #860

Conversation

besserox commented Mar 8, 2016

hpcugentbot commented Mar 8, 2016

boegel commented Mar 8, 2016

boegel Mar 8, 2016

Choose a reason for hiding this comment

hpcugentbot commented Mar 8, 2016

boegel Mar 8, 2016

Choose a reason for hiding this comment

besserox commented Mar 8, 2016

hpcugentbot commented Mar 8, 2016

boegel Mar 9, 2016

Choose a reason for hiding this comment

boegel commented Mar 9, 2016

hpcugentbot commented Mar 9, 2016

besserox commented Mar 9, 2016

boegel commented Mar 9, 2016

boegel commented Mar 9, 2016

boegel Mar 9, 2016

Choose a reason for hiding this comment

besserox Mar 10, 2016

Choose a reason for hiding this comment

besserox commented Mar 10, 2016

hpcugentbot commented Mar 10, 2016

boegel commented Mar 14, 2016

boegel commented May 10, 2016

besserox commented May 12, 2016

boegel commented May 13, 2016

boegel commented May 13, 2016