包测试失败，fatal error: stdlib.h: No such file or directory #4596
Comments
Title: Package test failed, fatal error: stdlib.h: No such file or directory |
|
|
|
|
破案了，有人修改了 zlib 包，这里导致间接使用系统库 system::z ：
但是我看 boost xmake.lua 里写的是 为什么会走 system 上去了。。。 |
The case was solved. Someone modified the zlib package, which resulted in the indirect use of system libraries: But I see that what is written in boost xmake.lua is Why did you go to system? . . |
知道了，system 默认是 true。。。 |
Got it, system defaults to true. . . |
|
@xq114 走 system:: 找，是最后的 fallback 方式，会去直接找 /usr/lib 我记得默认走 pkg-config 就能找到 zlib 的 |
@xq114 Go through system:: search, which is the final fallback method. It will directly search for /usr/lib I remember that by default, you can find zlib by running pkg-config. |
|
system::xxx 应该按照编译器探测才对，探测系统上编译器本来就能找到的包，不能加-isystem 之类flag，编译器自己找不到的包按不能用算。就像上面给的链接，对系统已有的路径加-isystem，那作用是去掉这个搜索路径，xmake不能这么加。 https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html 按 要么用parse cpp输出替换xmake里hardcode的includedirs，要么直接生成一段cpp程序尝试编译一下是否通过（直接链接要用的库），不走xmake自己的文件系统查找程序了。这么做还有一个好处，用户现在可以通过cflags来控制system::xxx的查找，这样对于手动安装、没有pkgconfig/cmake文件的包，也不用在xmake文件里硬编码路径 https://github.com/xmake-io/xmake/blob/master/xmake/modules/package/manager/system/find_package.lua 对于在系统上安装了却不在编译器搜索路径里的包，我认为应该无视而不是强行链接它们，因为不在编译器搜索路径说明人为进行了隔离，要么是编译器版本不匹配，要么是不希望用到这些包而隔离的。
这里zlib换成pkg-config当然可以，但不解决根本问题，其他使用system::xxx 找到的包还是不能用 |
system::xxx should be detected according to the compiler. It detects packages that the compiler can already find on the system. Flags such as -isystem cannot be added. Packages that the compiler cannot find by itself cannot be used. Just like the link given above, adding -isystem to the existing path in the system will remove the search path, which xmake cannot add. https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html According to the output of Either use the parse cpp output to replace the includedirs of the hardcode in xmake, or directly generate a cpp program and try to compile it to see if it passes (directly linking the library to be used), without using xmake's own file system search program. Another advantage of this is that users can now control the search for system::xxx through cflags. This way, for packages that are manually installed and do not have pkgconfig/cmake files, there is no need to introduce the hard-coded path https://github.com/xmake-io/xmake/blob/master/xmake/modules/package/manager/system/find_package.lua For packages that are installed on the system but are not in the compiler search path, I think they should be ignored rather than forcibly linked, because not being in the compiler search path means artificial isolation, either the compiler version does not match, or it is not expected Isolated using these packages.
It is of course possible to replace zlib with pkg-config here, but it does not solve the fundamental problem. Other packages found using system::xxx still cannot be used. |
|
One of my user had this issue on Linux, because zlib was found on system (system::z): but adding I think include and linkdirs shouldn't be added except when cross-compiling, something already done for xcodedirs (in _find_package_from_xcodedirs): if result then
-- we don't need to add linkdirs again if we are building target on the current platform (with -isysroot)
if config.plat() ~= opt.plat or config.arch() ~= opt.arch then
result.linkdirs = linkdirs
result.includedirs = includedirs
end
end-- 我的一个用户在 Linux 系统上遇到了这个问题，因为在系统 (system::z) 中找到了 zlib： 但在 gcc 命令行中添加 我认为，除非在交叉编译时，否则不应该添加 include 和 linkdirs，xcodedirs 已经这样做了（在 _find_package_from_xcodedirs 中）： if result then
-- we don't need to add linkdirs again if we are building target on the current platform (with -isysroot)
if config.plat() ~= opt.plat or config.arch() ~= opt.arch then
result.linkdirs = linkdirs
result.includedirs = includedirs
end
end |
macos 上的输出找不到 linkdirs $ cpp -v /dev/null -o /dev/null
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: x86_64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx14.0.0 -Wundef-prefix=TARGET_OS_ -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -E -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name null -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-strict-return -ffp-contract=on -fno-rounding-math -funwind-tables=2 -target-sdk-version=14.0 -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -debugger-tuning=lldb -target-linker-version 1015.6 -v -fcoverage-compilation-dir=/Users/ruki/projects/qiyi/harmony/iqiyi_harmony -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -Wno-elaborated-enum-base -Wno-reserved-identifier -Wno-gnu-folding-constant -fdebug-compilation-dir=/Users/ruki/projects/qiyi/harmony/iqiyi_harmony -ferror-limit 19 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcommon -fcolor-diagnostics -traditional-cpp -clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation -fno-odr-hash-protocols -clang-vendor-feature=+enableAggressiveVLAFolding -clang-vendor-feature=+revert09abecef7bbf -clang-vendor-feature=+thisNoAlignAttr -clang-vendor-feature=+thisNoNullAttr -mllvm -disable-aligned-alloc-awareness=1 -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o - -x c /dev/null
clang -cc1 version 15.0.0 (clang-1500.0.40.1) default target x86_64-apple-darwin23.0.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
# 1 "/dev/null"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 384 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "/dev/null" 2
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: x86_64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang: warning: argument unused during compilation: '-traditional' [-Wunused-command-line-argument]
可以尝试下 那这种呢 #1776 之前你提的搜索用户自定义路径，这么搞，编译器也能找到？ |
可以的，本来那几个变量就是编译器会去额外寻找的环境变量 |
Yes, those variables are originally environment variables that the compiler will look for additionally. |
|
试试，通过 check_cxsnippets 的方式探测，参数用法和它一致。configs 里面可以传 defines/cflags/links等等 如果尝试编译检测通过，configs 里面的所有配置，都会作为 find_package 结果返回。。 package:find_package("system::z", {includes = "zlib.h", configs = {defines = "TEST"}})
package:find_package("system::zlib", {includes = "zlib.h", configs = {links = "z"}})
另外，还可以传 funcs/snippets 参数，严格定制化检测 |
Try it and detect it through check_cxsnippets. The parameter usage is consistent with it. Defines/cflags/links, etc. can be passed in configs If the attempted compilation test passes, all configurations in configs will be returned as find_package results. . package:find_package("system::z", {includes = "zlib.h", configs = {defines = "TEST"}})In addition, you can also pass funcs/snippets parameters to strictly customize the detection. |
|
再试试，应该可以了 xmake update -s dev |
Try again, it should work xmake update -s dev |
可以用一下xmake-requires.lock这个功能，我最近刚刚用上，see https://github.com/XmacsLabs/mogan/blob/branch-1.2/xmake-requires.lock |
You can use the xmake-requires.lock function. I just used it recently, see https://github.com/XmacsLabs/mogan/blob/branch-1.2/xmake-requires.lock |
我之前用过，但是在我们的环境发现一些 bug。用户编译，不知道为什么，导致 xmake-requires.lock 文件被 xmake 修改掉，文件全乱了。我暂时没法用最小工程复现它。 |
I've used it before, but found some bugs in our environment. When compiled by the user, for some unknown reason, the xmake-requires.lock file was modified by xmake and the files were all messed up. I can't reproduce it with minimal project yet. |
目前测试 ok 了 |
The current test is ok |
现在 system/find_package 里面去调用编译器检测，会有个问题，find_package 是在 package:fetch 阶段就会被执行。。这个时候使用 package toolchain，得先 check package toolchain 目前这个 check 仅仅是在 install 阶段才会去做，所以会导致一些 toolchain 在 fetch 阶段检测失败被缓存，后期安装时候，也会失败。。例如 link.exe 前期检测失败，干扰了后期 clang-cl/link 安装 #4859 但如果将 check package toolchain 提前到 fetch 阶段做，有些包是 toolchain 包，有依赖关系，都还没安装上，会检测不到，也不行。 |
Now there will be a problem when calling compiler detection in system/find_package. find_package will be executed during the package:fetch stage. . To use package toolchain at this time, you must first check package toolchain Currently, this check is only done during the install phase, so some toolchains may fail to be detected during the fetch phase and be cached, and may also fail during later installations. . For example, the early detection of link.exe failed, which interfered with the later installation of clang-cl/link #4859 However, if you move the check package toolchain to the fetch stage in advance, some packages are toolchain packages and have dependencies that have not been installed yet, so they will not be detected and will not work. |
理论上一个包on_fetch就是应该带toolchain信息的，不同的toolchain能用的包都不一样，除了本身就是toolchain类型的包。能不能在添加package的时候单独处理toolchain？先安装/加载toolchain类型，完成之后再绑定toolchain的环境变量去安装/加载别的包？ （其实对于remote toolchain我更倾向于把toolchain单独写到本地env.lua里而不是写到xmake.lua里，用xrepo env shell加载之后再跑xmake。toolchain本来就应该是各个机器有自己的toolchain版本，xmake.lua不应该完全定死一个toolchain） |
我知道，但是现在不太好实现 fetch 阶段使用 toolchain。。关键是要同时支持 其他包依赖 toolchain 包的情况。。 比如设置了 如果先初始化 package 和 fetch ，就会遇到当前 fetch 阶段调用 toolchain find_package 失败的问题。 除非是搞成，llvm 这些 toolchain 包，先全部安装好，之后再安装其他包，这个逻辑就更复杂了。 不过目前我稍微做了折中，暂时应该没问题了。
走 另外，都在 xmake.lua 对于一些新手，可以实现一键 xmake 编译，自动拉取 toolchain 和使用，可以有效降低使用门槛，避免趟坑。。不可能每个工程都去 readme 里面写一堆，如果对不同 toolchain ，去配置 xrepo env 加载编译，用户估计看到这么复杂，直接吓跑了。 |
I know, but it's not easy to implement the fetch phase using toolchain now. . The key is to also support the situation where other packages depend on the toolchain package. . For example, if If you initialize package and fetch first, you will encounter the problem of failure to call toolchain find_package in the current fetch stage. Unless it is done, all the llvm toolchain packages must be installed first, and then other packages can be installed. This logic will be more complicated. But now I've made a slight compromise and it should be fine for the time being.
Whether to use In addition, it is all in xmake.lua. For some novices, they can realize one-click xmake compilation, automatically pull the toolchain and use it, which can effectively lower the threshold of use and avoid pitfalls. . It is impossible to write a lot of information in the readme for every project. If you configure xrepo env to load and compile different toolchains, users will probably be scared away when they see such complexity. |
Xmake 版本
xmake v2.8.3+dev.f374224
操作系统版本和架构
Linux 10-16-43-32 5.14.0-70.13.1.el9_0.x86_64 #1 SMP PREEMPT Wed May 25 21:01:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
描述问题
这两天突然遇到的问题（ci 全挂了），在这之前，都能正常安装包。应该没有更新过 xmake。
比较奇怪的是，在另一个环境测试是没有问题的，经过比对发现，能安装成功的那个，是没有 -isystem /usr/include 选项的，见下图
请您帮忙看一下，可能是哪里的问题。虽然是不同的运行环境，但是我们都是使用统一的 docker 镜像进行编译，docker 环境是完全一致的。
关于 -isystem 的副作用，在这里有介绍：https://stackoverflow.com/questions/37218953/isystem-on-a-system-include-directory-causes-errors
期待的结果
包测试可正常 work
工程配置
boost 包的 xmake.lua
附加信息和错误日志
The text was updated successfully, but these errors were encountered: