Bootstrap fails on CentOS 6

[mrobbert]

I am trying to install Easybuild on CentOS 6 and can not get it to work. I tried first on our cluster and after multiple unsuccesful attempts I installed a basic CentOS 6 VM to see if maybe we had some customizations that were incompatible, but I'm seeing the same error. Should this work or do we need to upgrade to CentOS 7?

Here are my systems stats:

[mrobbert@centos6 ~]$ python -V
Python 2.6.6
[mrobbert@centos6 ~]$ type module
module is a function
module ()
{
    eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTARG_CMD:-:} -s sh)
}
[mrobbert@centos6 ~]$ type -f module
-bash: type: module: not found
[mrobbert@centos6 ~]$ module --version

Modules based on Lua: Version 7.7.15  2018-02-13 14:12 -07:00
    by Robert McLay mclay@tacc.utexas.edu

[mrobbert@centos6 ~]$ module av EasyBuild
No modules found!
Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

[mrobbert@centos6 ~]$ which -a eb
/usr/bin/which: no eb in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/mrobbert/bin)
[mrobbert@centos6 ~]$ eb --version
-bash: eb: command not found
[mrobbert@centos6 ~]$

Here is the full output of the bootstrap:

[mrobbert@centos6 ~]$ python bootstrap_eb.py /sw
[[INFO]] EasyBuild bootstrap script (version 20180201.01, MD5: 5211e214b622e6d80614c3127eec2396)
[[INFO]] Found Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) ; [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]

[[INFO]] Installation prefix /sw
[[INFO]] Found module command '/sw/apps/lmod/lmod/libexec/lmod' via $LMOD_CMD (Lmod), so using it.
[[INFO]] No suitable setuptools installation found, proceeding with stage 0...


[[INFO]] +++ STAGE 0: installing distribute via included (patched) distribute_setup.py...

Downloading https://easybuilders.github.io/easybuild/files/distribute-0.6.49-patched1.tar.gz
Extracting in /tmp/tmpgeoF_c
Now working in /tmp/tmpgeoF_c/distribute-0.6.49
Installing Distribute
[[INFO]] Installed setuptools version 0.6 (/tmp/tmpIbOEwo/lib/python2.6/site-packages/distribute-0.6.49-py2.6.egg/setuptools/__init__.pyc)


[[INFO]] +++ STAGE 1: installing EasyBuild in temporary dir with easy_install...

[[INFO]] installing EasyBuild with 'easy_install --quiet --upgrade --prefix=/tmp/tmpIbOEwo/eb_stage1 easybuild'
[[ERROR]] Running 'easy_install --quiet --upgrade --prefix=/tmp/tmpIbOEwo/eb_stage1 easybuild' failed: error: Could not find suitable distribution for Requirement.parse('easybuild')
Traceback (most recent call last):
  File "bootstrap_eb.py", line 359, in run_easy_install
    easy_install.main(args)
  File "/tmp/tmpIbOEwo/lib/python2.6/site-packages/distribute-0.6.49-py2.6.egg/setuptools/command/easy_install.py", line 1973, in main
    with_ei_usage(lambda:
  File "/tmp/tmpIbOEwo/lib/python2.6/site-packages/distribute-0.6.49-py2.6.egg/setuptools/command/easy_install.py", line 1954, in with_ei_usage
    return f()
  File "/tmp/tmpIbOEwo/lib/python2.6/site-packages/distribute-0.6.49-py2.6.egg/setuptools/command/easy_install.py", line 1977, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "/usr/lib64/python2.6/distutils/core.py", line 169, in setup
    raise SystemExit, "error: " + str(msg)
SystemExit: error: Could not find suitable distribution for Requirement.parse('easybuild')

[boegel]

@mrobbert This issue has been reported before, the problem seems to be that old versions of setuptools can not communicate with https://pypi.python.org/pypi anymore... See https://bugzilla.redhat.com/show_bug.cgi?id=1510444 for more details.

The easiest workaround is to update your system setuptools installation to a sufficiently recent version, that way the bootstrap script won't need to install an old version itself...

[boegel]

A fix we have in mind for this is to try pip as well in case easy_install fails during stage 1, see #2389.

Would that help in this case @mrobbert?

[mrobbert]

I can't say if that fix would work or not, but if it gave me a path to install on CentOS or RHEL 6.x that would help me and I'd be willing to test it.
I will note that I did a little more playing around in my VM and found that if I ran the easy_install manually it failed due to a problem with git+https being an unknown protocol. Then I tried the offline install procedures and that worked. I thought I had done that on the production cluster earlier and it didn't work, but I may need to revisit that.

[boegel]

@mrobbert Do you have openssl-devel installed? Cfr. easybuilders/easybuild#23

I'll try and see if I can find some time to enhance the bootstrap script to fall back to pip.

I guess the first questions there are: i) do you have pip installed, ii) which version?

[mrobbert]

[mrobbert@centos6 ~]$ rpm -qa|grep openssl
openssl-devel-1.0.1e-57.el6.x86_64
openssl-1.0.1e-57.el6.x86_64
[mrobbert@centos6 ~]$ rpm -qa|grep pip
python-pip-7.1.0-1.el6.noarch

[boegel]

@mrobbert Sorry for not getting back on this earlier.

I did look into letting the bootstrap script use pip, but it turned out that's not trivial, partially due to the fact that pip doesn't properly install namespace packages (which is required by both vsc-base and the easybuild-* packages), see pypa/pip#1924 .

There is a workaround however (I should have mentioned this earlier...): you can do an offline installation, by re-downloading the necessare source tarballs for the vsc-install, vsc-base and easybuild-* Python packages manually, and then letting the bootstrap script use those, see also https://easybuild.readthedocs.io/en/latest/Installation.html#offline-bootstrapping-using-supplied-source-tarballs .

For example:

# download source tarballs from PyPI
curl -OL https://files.pythonhosted.org/packages/18/59/3274a58af6af84a87f7655735b452c06c769586ee73954f5ee15d303aa29/vsc-install-0.11.3.tar.gz
curl -OL https://files.pythonhosted.org/packages/62/e5/589612e47255627e4752d99018ae7cff8f49ab0fa6b4ba7b2226a76a05d3/vsc-base-2.8.3.tar.gz
curl -OL https://files.pythonhosted.org/packages/c0/3a/88c89e39887487719786fb9ad39675443ddff5b0d2aee7fc88bd3daf83ba/easybuild-framework-3.8.0.tar.gz
curl -OL https://files.pythonhosted.org/packages/c9/7d/6b02af21ad58679ea6c6ec51ca857f1d425e9f5d3be81137a6fea57b1211/easybuild-easyblocks-3.8.0.tar.gz
curl -OL https://files.pythonhosted.org/packages/de/9e/46710d49342da1c7ba722131d8a33b167f7f1dfa6ffd8184fbe303a6824a/easybuild-easyconfigs-3.8.0.tar.gz
# specify location of pre-downloaded source tarballs for offline bootstrapping
export EASYBUILD_BOOTSTRAP_SOURCEPATH=$PWD
# specify location where to install EasyBuild (also specifies to EasyBuild where it should install software)
export EASYBUILD_PREFIX=/tmp/$USER/test123
# perform (offline) bootstrap
python bootstrap_eb.py $EASYBUILD_PREFIX

[boegel]

I hope that the issues with the bootstrap script can be resolved once and for all with EasyBuild 4.0 which we hope to release later this year. Two large goals there are to make it stand-alone (no required dependencies other than Python standard library) and to have single-tarball releases. Both of those will help a lot with resolving installation problems like this one, especially since it implies that we don't need to use namespacing anymore (so compatibility with pip is also covered).

[runiq]

I think I am hitting this error. IIUC I need to install a newer version of setuptools than the one that stage0 uses, right?

[boegel]

@runiq Yes, having a sufficiently recent setuptools already installed should be sufficient.

For the upcoming EasyBuild 4.0, setuptools will no longer be a runtime requirement (I need to check whether we can also avoid making it an installation requirement).