WIP: Switch to devtoolset-4

[jakirkham]

This PR proposes that we switch to devtoolset-4. It is needed to package libdynd. The devtoolset provides C++14 support and gcc 5.2.1. Just like devtoolset-2, we will still be able to ship to any CentOS 6 machine without needing to include standard system libraries. There is no change to our GLIBC compatibility. In the long run, it sounds like we want to package gcc. However, this should hold us over until then.

cc @msarahan @jjhelmus @pelson @ocefpaf @izaid

[msarahan]

OK with me as an interim solution.

[wesm]

Does devtoolset-4 avoid issues w/ the gcc 5 ABI (for example, a std::string passed from something compiled with devtoolset-2 will be OK)?

[izaid]

@wesm My understanding is that behavior can be controlled by a compile-time flag, details at https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

[msarahan]

Sure, but what is the default?

On Mon, Jul 18, 2016, 11:55 Irwin Zaid notifications@github.com wrote:

@wesm https://github.com/wesm My understanding is that behavior can be
controlled by a compile-time flag, details at
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#22 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACV-Z0loWIwfZRezIdUtEwa0Wg6TMkoks5qW7AMgaJpZM4JOSXD
.

[izaid]

@msarahan Default is the new ABI, old ABI is available via -D_GLIBCXX_USE_CXX11_ABI=0. See http://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c11-abi/

[jakirkham]

That's about the compiler generally not the devtoolset. I would figure since they are trying to allow things to just work on CentOS 6, they would configure the compiler to use the old ABI. Though I can't find a ref ATM.

If it is forcing the ABI to the new version, we may have found the controversial point with this interim solution. How do you feel about devtoolset-3, @izaid? That provides gcc version 4.9.2 FWICT.

[izaid]

@jakirkham devtoolset-3 should be fine, as we support GCC 4.9, but would want to try to be sure.

And, yeah, I can't find the old one being the default either.

[jakirkham]

So the docs I found suggest that they break compatibility when C++11 or C++14 is used.

I suppose one could add the needed flag to the toolchain to avoid this problem, but that only works when people use the toolchain. 😒

Maybe devtoolset-3 is the way to go. I'll update this.

[jakirkham]

See this PR ( #23 ) for the devtoolset-3 alternative. I have verified that it provides gcc version 4.9.2. Also have copied the same people and everyone in the discussion here on that PR.

[jjhelmus]

If this breaks compatibility with older packages it will probably requires that conda-forge recompile a number of packages. This can be done but I think a choice like this should be discussed during a conda-forge meeting. Additional a few tests cases could be examined how the ABI incompatibility would exhibit itself to users. devtoolset-3 seems less controversial.

[jakirkham]

If this breaks compatibility with older packages it will probably requires that conda-forge recompile a number of packages. This can be done but I think a choice like this should be discussed during a conda-forge meeting.

👍 Hence PR ( #23 ).

If we provide that flag -D_GLIBCXX_USE_CXX11_ABI=0 consistently, we should be able to avoid breaks. However, the consistent part concerns me as we are already debating about how to do things consistently.

This all being said, we will need to have this discussion eventually because we are discussing adding a gcc package from 5.x in this PR ( conda-forge/staged-recipes#873 ) for the long term. Also we have been discussing having OS X and Linux both use that gcc. That requires a total rebuild of OS X and possibly Linux. I'd rather wait to do a total rebuild until we have that compiler.

Additional a few tests cases could be examined how the ABI incompatibility would exhibit itself to users.

My understanding is they made some changes to std::string and std::list to comply with Big-O guarantees of C++11 that the 4.x series could not have done. Though 👍 for more investigation.

devtoolset-3 seems less controversial.

Agreed.

[insertinterestingnamehere]

Small follow on to this, it appears that the docs for the devtoolset (already linked above) say it uses the old ABI for the time being. "libstdc++ now has full support for the C++11 standard with the exception of the new implementations of the std::string and std::list classes because the Red Hat Developer Toolset 4.1 version of GCC uses the old ABI." The issues they are discussing with c++11 and c++14 are about how they will likely change to the newer ABI in a future release. Because of the gcc ABI change, they can't guarantee the binaries built now will be compatible with binaries built with a later major release of devtoolset for C++ 11 and 14. That aside, gcc 4.9 should be good enough for libdynd for the time being, so that's fine too.

[jakirkham]

What section was it? I seem to be directed to the legal notice. Though I'm on my phone so that could be part of the problem.

[insertinterestingnamehere]

The part warning about future ABI incompatibilities is "C++ ABI." The clarification about which ABI is used by default is in the section "Changes Since Red Hat Developer Toolset 3.1."

[insertinterestingnamehere]

One potential concern with devtoolset 4 is that it's gcc 5.3 whereas Continuum's libgcc package only comes from gcc 5.2. It seems like it would be an easy fix to the libgcc package there though.
@msarahan any thoughts? This a one case where ContinuumIO/anaconda-issues#839 actually becomes relevant.

[msarahan]

I'd like to revisit this. Given that the docs: https://access.redhat.com/documentation/en-US/Red_Hat_Developer_Toolset/4/html-single/User_Guide/index.html#sect-GCC-CPP-Compatibility state that the old ABI is used with devtoolset-4. I'm pretty certain that devtoolset-4 would be completely compatible. libgcc should not be an issue because the devtoolset does its squirrelly partial static linking.

I recommend that this PR be merged, and that #23 be closed.

[msarahan]

PS, the section of the docs that convinced me:

The default language standard setting for Red Hat Developer Toolset is C++98. Any C++98-compliant binaries or libraries built in this default mode (or explicitly with -std=c++98) can be freely mixed with binaries and shared libraries built by the Red Hat Enterprise Linux 6 or 7 system toolchain GCC.

[jakirkham]

That doesn't say anything about C++11 though. Presumably things built with C++11 would be broken by the ABI change. Though I don't know as I'm still unclear.

[jakirkham]

@ncoghlan, we are a bit confused about the default C++ ABI used by devtoolset-4 and whether it is using the C++11 ABI breaks made in GCC 5.x by default or not. We would really appreciate if you could advise us as to how to proceed. Would using devtoolset-3 be a safer option? Can we use devtoolset-4 safely somehow? Note we are using devtoolset-2 currently. Thanks.

[ncoghlan]

I read the devtoolset-4 docs the same way @msarahan does: they use -std=C++98 by default, so as long as you're building for RHEL 6 or higher, you shouldn't have any problems.

You may run into trouble with folks wanting to use conda build --wheel though: conda/conda-build#1330

The issue there is that manylinux1 still uses RHEL/CentOS 5 as the de facto cross-distro ABI baseline, and devtoolset-2 was the last version to support that. That means going to a later devtoolset version means implicitly embarking on the task of defining a RHEL/CentOS 6 based manylinux2. While that definitely needs to happen at some point (as RHEL/CentOS 5 officially go EOL next year), it should probably be an explicit decision rather than an inadvertent side effect of a toolchain upgrade on conda-forge.

[jjhelmus]

conda-forge is using a CentOS 6 base for its Docker container so any wheels which were built would likely not meet the manylinux1 standard. I do not foresee conda-forge switching to a CentOS 5 container as the choice to use CentOS 6 was made explicitly some time ago. This may cause issue if conda-forge decides to start producing wheels but I think this is still a ways off and can be dealt with at a later time.

I would like to see the suggestion to move to devtoolset-4 submitted as a conda-forge enhancement proposal so that the technical merits and challenges can be examined.

[ncoghlan]

OK, if you're already using a CentOS 6 base, then I don't foresee any new problems arising from switching to devtoolset-4 as your build chain.

For the wheel side of things (assuming you eventually decide to go that way), even when manylinux1 was first defined, the assumption was that there would eventually be a manylinux2 that bumped the baseline ABI version up another few years, it was just an open question as to whether the trigger for that would be CentOS 5 going EOL in 2017, or if someone would want it earlier than that.

[jakirkham]

Where should we note our interest in a CentOS 6 based manylinux2? Is there somewhere we can raise an issue?

[jakirkham]

I would like to see the suggestion to move to devtoolset-4 submitted as a conda-forge enhancement proposal so that the technical merits and challenges can be examined.

To be honest, I would almost rather wait and see how manylinux2 and/or a conda-packaged compiler shake out. Though the latter really deserves a CFEP too.

[jakirkham]

Closing this out as we will be moving to Anaconda's new compilers after moving to conda-build 3.