Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug] AutotoolsToolchain uses the build compiler rather than host compiler #13219

Closed
sfackler opened this issue Feb 24, 2023 · 5 comments · Fixed by #13230
Closed

[bug] AutotoolsToolchain uses the build compiler rather than host compiler #13219

sfackler opened this issue Feb 24, 2023 · 5 comments · Fixed by #13230
Assignees
@jcar87
Milestone
2.0.2

Comments

@sfackler
Copy link
Contributor

Environment details

  • Operating System+version: M1 macOS 13.1
  • Compiler+version: N/A
  • Conan version: 2.0.0 and 1.59.0
  • Python version: 3.10.10

Steps to reproduce

  1. Define a profile that targets Linux (an actual cross toolchain is not needed):
[settings]
os=Linux
arch=x86_64
compiler=gcc
compiler.version=11

[buildenv]
CC=x86_64-linux-gnu-gcc
CXX=x86_64-linux-gnu-g++
AR=x86_64-linux-gnu-ar
LD=x86_64-linux-gnu-ld
  1. Create a simple test conanfile.py that uses the AutotoolsToolchain tool and dumps out its generated cflags:
from conan import ConanFile
from conan.tools.gnu import AutotoolsToolchain

class TestConan(ConanFile):
    settings = { "os", "arch" }
    def generate(self):
        tc = AutotoolsToolchain(self)
        tc.generate(tc.environment())

        raise Exception(tc.cflags)
  1. Run a build using the default build profile and cross host profile:
$ conan install --profile:build default --profile:host ./linux-cross --build=missing .

Logs

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
compiler=gcc
compiler.version=11
os=Linux
[buildenv]
CC=x86_64-linux-gnu-gcc
CXX=x86_64-linux-gnu-g++
AR=x86_64-linux-gnu-ar
LD=x86_64-linux-gnu-ld

Profile build:
[settings]
arch=armv8
build_type=Release
compiler=apple-clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=14
os=Macos


======== Computing dependency graph ========
Graph root
    conanfile.py: /Users/sfackler/test/conanfile.py

======== Computing necessary packages ========

======== Installing packages ========

======== Finalizing install (deploy, generators) ========
conanfile.py: Calling generate()
conanfile.py: Generators folder: /Users/sfackler/test
ERROR: conanfile.py: Error in generate() method, line 10
        raise Exception(tc.cflags)
        Exception: ['-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk', '-arch x86_64']

Note that the resolved cflags contain flags meant for clang while the host profile indicates the compiler is gcc. It is picking up some of the host profile though since it's setting the architecture to x86_64 rather than the build profile's armv8.

This is a stripped down reproduction of a failure to cross compile the openssl recipe on macOS that fails due to the same introduction of clang cflags to the GCC compiler.
@sfackler
Copy link
Contributor Author

Here's the full output of trying to build openssl/1.1.1t using a real GCC cross compilation toolchain:

Configuration (profile_host):
[settings]
arch=armv8
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[options]
cross-toolchain*:target=aarch64-ubuntu-18.04
[build_requires]
*: cross-toolchain/0.24.0-1-gddcf805
[env]

Configuration (profile_build):
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=apple-clang
compiler.libcxx=libc++
compiler.version=14
os=Macos
os_build=Macos
[options]
[build_requires]
[env]

openssl/1.1.1t: WARN: Package binary is corrupted, removing: 115c8fa606e08a868a5ec8073c26c1a096c7263b
conanfile.txt: Installing package
Requirements
    openssl/1.1.1t from 'conancenter' - Cache
Packages
    openssl/1.1.1t:115c8fa606e08a868a5ec8073c26c1a096c7263b - Build
Build requirements
    cross-toolchain/0.24.0-1-gddcf805 from local cache - Cache
Build requirements packages
    cross-toolchain/0.24.0-1-gddcf805:c9a9e1b81012dda83d7aae6594d79a1d6b36445a - Cache

Cross-build from 'Macos:armv8' to 'Linux:armv8'
Installing (downloading, building) binaries...
cross-toolchain/0.24.0-1-gddcf805: Already installed!
openssl/1.1.1t: Applying build-requirement: cross-toolchain/0.24.0-1-gddcf805
openssl/1.1.1t: WARN: Build folder is dirty, removing it: /Users/sfackler/.conan/data/openssl/1.1.1t/_/_/build/115c8fa606e08a868a5ec8073c26c1a096c7263b
openssl/1.1.1t: Copying sources to build folder
openssl/1.1.1t: Building your package in /Users/sfackler/.conan/data/openssl/1.1.1t/_/_/build/115c8fa606e08a868a5ec8073c26c1a096c7263b
openssl/1.1.1t: Generator txt created conanbuildinfo.txt
openssl/1.1.1t: Calling generate()
openssl/1.1.1t: Aggregating env generators
openssl/1.1.1t: Calling build()
openssl/1.1.1t: Apply patch (portability): TVOS and WatchOS don't like fork()
openssl/1.1.1t: gen_info = {'CFLAGS': ['-fPIC', '-O3', '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk', '-arch arm64'], 'CXXFLAGS': ['-fPIC', '-O3', '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk', '-arch arm64'], 'DEFINES': ['NDEBUG'], 'LDFLAGS': ['-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk', '-arch arm64']}
openssl/1.1.1t: using target: conan-Release-Linux-armv8-gcc-11 -> linux-aarch64
openssl/1.1.1t: my %targets = (
    "conan-Release-Linux-armv8-gcc-11" => {
        inherit_from => [ "linux-aarch64" ],
        cflags => add("-fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64"),
        cxxflags => add("-fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64"),
        defines => add(["NDEBUG"]),
        includes => add(),
        lflags => add("-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64"),








    },
);

openssl/1.1.1t: ['"conan-Release-Linux-armv8-gcc-11"', 'no-shared', '--prefix=/', '--openssldir="/etc/ssl"', 'no-unit-test', 'threads', 'PERL=perl', 'no-tests', '--release', '--libdir=lib', '-fPIC', 'no-md2']
Configuring OpenSSL version 1.1.1t (0x1010114fL) for conan-Release-Linux-armv8-gcc-11
Using os-specific seed configuration
Creating configdata.pm
Creating Makefile

**********************************************************************
***                                                                ***
***   OpenSSL has been successfully configured                     ***
***                                                                ***
***   If you encounter a problem while building, please open an    ***
***   issue on GitHub <https://github.com/openssl/openssl/issues>  ***
***   and include the output from the following command:           ***
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL file first)         ***
***                                                                ***
**********************************************************************
perl "-I." -Mconfigdata "util/dofile.pl" \
            "-oMakefile" include/crypto/bn_conf.h.in > include/crypto/bn_conf.h
perl "-I." -Mconfigdata "util/dofile.pl" \
            "-oMakefile" include/crypto/dso_conf.h.in > include/crypto/dso_conf.h
perl "-I." -Mconfigdata "util/dofile.pl" \
            "-oMakefile" include/openssl/opensslconf.h.in > include/openssl/opensslconf.h
/Library/Developer/CommandLineTools/usr/bin/make depend && /Library/Developer/CommandLineTools/usr/bin/make _all
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/app_rand.d.tmp -MT apps/app_rand.o -c -o apps/app_rand.o apps/app_rand.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/apps.d.tmp -MT apps/apps.o -c -o apps/apps.o apps/apps.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/bf_prefix.d.tmp -MT apps/bf_prefix.o -c -o apps/bf_prefix.o apps/bf_prefix.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/opt.d.tmp -MT apps/opt.o -c -o apps/opt.o apps/opt.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/s_cb.d.tmp -MT apps/s_cb.o -c -o apps/s_cb.o apps/s_cb.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF apps/s_socket.d.tmp -MT apps/s_socket.o -c -o apps/s_socket.o apps/s_socket.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF crypto/aes/aes_cbc.d.tmp -MT crypto/aes/aes_cbc.o -c -o crypto/aes/aes_cbc.o crypto/aes/aes_cbc.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF crypto/aes/aes_cfb.d.tmp -MT crypto/aes/aes_cfb.o -c -o crypto/aes/aes_cfb.o crypto/aes/aes_cfb.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF crypto/aes/aes_core.d.tmp -MT crypto/aes/aes_core.o -c -o crypto/aes/aes_core.o crypto/aes/aes_core.c
aarch64-linux-gnu-gcc  -I. -Iinclude -fPIC -pthread -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -Wa,--noexecstack -fPIC -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -arch arm64 -fPIC -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/etc/ssl\"" -DENGINESDIR="\"//lib/engines-1.1\"" -DNDEBUG -DNDEBUG -DNDEBUG -MMD -MF crypto/aes/aes_ecb.d.tmp -MT crypto/aes/aes_ecb.o -c -o crypto/aes/aes_ecb.o crypto/aes/aes_ecb.c
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [apps/app_rand.o] Error 1
make[1]: *** Waiting for unfinished jobs....
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [apps/bf_prefix.o] Error 1
make[1]: *** [apps/s_cb.o] Error 1
make[1]: *** [apps/opt.o] Error 1
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [crypto/aes/aes_ecb.o] Error 1
make[1]: *** [apps/apps.o] Error 1
make[1]: *** [apps/s_socket.o] Error 1
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [crypto/aes/aes_cfb.o] Error 1
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [crypto/aes/aes_core.o] Error 1
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
aarch64-linux-gnu-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make[1]: *** [crypto/aes/aes_cbc.o] Error 1
make: *** [all] Error 2
openssl/1.1.1t:
openssl/1.1.1t: ERROR: Package '115c8fa606e08a868a5ec8073c26c1a096c7263b' build failed
openssl/1.1.1t: WARN: Build folder /Users/sfackler/.conan/data/openssl/1.1.1t/_/_/build/115c8fa606e08a868a5ec8073c26c1a096c7263b/build-release
ERROR: openssl/1.1.1t: Error in build() method, line 724
        autotools.make()
        ConanException: Error 2 while executing make -j10

@sfackler
Copy link
Contributor Author

It looks like this needs to look at os_host, not os_build:

conan/conan/tools/gnu/autotoolstoolchain.py

Line 79 in 7631fbb

if os_build == "Macos":

@sfackler sfackler mentioned this issue Feb 24, 2023
Merged
5 tasks
@jcar87
Copy link
Contributor

jcar87 commented Feb 24, 2023

Hi @sfackler - thanks so much for trying this out on both Conan 2.0 and 1.59 and for your thorough investigation.

Indeed I believe there could be a bug in that particular line. Will review your PR to fix this, thanks for raising it!

@sfackler
Copy link
Contributor Author

sfackler commented Mar 7, 2023

@jcar87 Are you going to have a chance to review the PR?

@jcar87 jcar87 self-assigned this Mar 7, 2023
@jcar87
Copy link
Contributor

jcar87 commented Mar 14, 2023 •
edited
Loading

Hi @sfackler - apologies for the delay in getting to this. We have reviewed it and added a couple of fixes and a unit test to avoid regressions in the future. Thanks so much for the investigation and your proposed fix, it was spot on :)
The PR will be merged once it passes our CI validation and the fix will be part of the upcoming Conan 2.0.2 release.

@memsharded memsharded added this to the 2.0.2 milestone Mar 14, 2023
@jcar87 jcar87 mentioned this issue Mar 14, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jcar87 jcar87

Labels
None yet
Projects
None yet
Milestone
2.0.2
Development

Successfully merging a pull request may close this issue.

Fix macOS <-> Linux cross cflags resolution sfackler/conan
3 participants
@sfackler @jcar87 @memsharded