Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create static library providing C++ module #5359

Open
iDingDong opened this issue Jul 18, 2024 · 14 comments
Open

Cannot create static library providing C++ module #5359

iDingDong opened this issue Jul 18, 2024 · 14 comments
Labels

Comments

@iDingDong
Copy link

Xmake Version

2.9.3

Operating System Version and Architecture

Windows 11 23H2

Describe Bug

I attempted to compile the following code with xmake and MSVC:

test.cppm

export module t1;

xmake.lua

target("testm")
  set_kind("static")
  set_languages("cxxlatest")
  add_files("./test.cppm")
target_end()

The command xmake failed with the following output:

checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33812
[  0%]: <testm> generating.module.deps test.cppm
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.ixx
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.compat.ixx
[ 50%]: <testm> compiling.bmi.release std
[ 66%]: <testm> compiling.bmi.release std.compat
[ 83%]: archiving.release testm.lib
error: LINK : fatal error LNK1181: cannot open input file 'build\.objs\testm\windows\x64\release\test.cppm.obj'

Expected Behavior

The code compiles and gives me testm.lib

Project Configuration

testsxm.zip

Additional Information and Error Logs

When compiling with `_vD' xmake outputs:

checking for platform ... windows
checking for architecture ... x64
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33812
checkinfo: cannot runv(zig.exe version), No such file or directory
checking for zig ... no
checkinfo: cannot runv(zig.exe version), No such file or directory
checking for zig ... no
checkinfo: cannot runv(nim.exe --version), No such file or directory
checking for nim ... no
checkinfo: cannot runv(nim.exe --version), No such file or directory
checking for nim ... no
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe
checking for the c++ compiler (cxx) ... cl.exe
checking for C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe ... ok
checking for flags (cl_scan_dependencies) ... ok
> cl.exe "-scanDependencies" "C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_3D6AA89633D343608A3B474C0A3F5760.json" "-nologo"
checking for flags (cl_ifc_output) ... ok
> cl.exe "-ifcOutput" "C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_9FE53270D1A34A50891A83A095BCD9A0" "-nologo"
[  0%]: <testm> generating.module.deps test.cppm
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\ac7906e3\test.cppm.module.json test.cppm -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\ac7906e3 -Fobuild\.objs\testm\windows\x64\release\test.cppm.obj
test.cppm
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.ixx
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ixx.module.json "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.ixx" -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab -Fobuild\.objs\testm\windows\x64\release\889e61fa4b2b4a678264d65ff7ac448b\std.ixx.obj
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.compat.ixx
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ixx.module.json "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.compat.ixx" -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab -Fobuild\.objs\testm\windows\x64\release\889e61fa4b2b4a678264d65ff7ac448b\std.compat.ixx.obj
std.ixx
std.compat.ixx
[ 50%]: <testm> compiling.bmi.release std
checking for flags (cl_interface) ... ok
> cl.exe "-interface" "-nologo"
checking for flags (cl_ifc_only) ... ok
> cl.exe "-ifcOnly" "-nologo"
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\cl.exe" -c -nologo -std:c++latest /EHsc -TP -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc -interface -ifcOnly -Fobuild\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.ixx"
checking for flags (cl_reference) ... ok
> cl.exe "-reference" "Foo=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_F8DF50D89A734400861D092D6AE01860" "-nologo"
checking for flags (cl_header_unit_quote) ... ok
> cl.exe "-std:c++latest" "-headerUnit:quote" "foo.h=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_ECE628A3379B43008C669A1D7BC05260" "-nologo"
checking for flags (cl_header_unit_angle) ... ok
> cl.exe "-std:c++latest" "-headerUnit:angle" "foo.h=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_ECE628A3379B43008C669A1D7BC05260" "-nologo"
[ 66%]: <testm> compiling.bmi.release std.compat
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\cl.exe" -c -nologo -std:c++latest /EHsc -reference std=build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc -TP -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ifc -interface -ifcOnly -Fobuild\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ifc "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.compat.ixx"
checking for link.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\link.exe
checking for the static library archiver (ar) ... link.exe
[ 83%]: archiving.release testm.lib
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\link.exe" -lib -nologo -machine:x64 -out:build\windows\x64\release\testm.lib build\.objs\testm\windows\x64\release\test.cppm.obj
error: @programdir\core\main.lua:329: @programdir\actions\build\main.lua:148: @programdir\modules\async\runjobs.lua:325: @programdir\actions\build\kinds\static.lua:53: @programdir\modules\core\tools\link.lua:175: LINK : fatal error LNK1181: cannot open input file 'build\.objs\testm\windows\x64\release\test.cppm.obj'

stack traceback:
    [C]: in function 'error'
    [@programdir\core\base\os.lua:973]:
    [@programdir\modules\core\tools\link.lua:175]: in function 'catch'
    [@programdir\core\sandbox\modules\try.lua:123]: in function 'try'
    [@programdir\modules\core\tools\link.lua:150]:
    [C]: in function 'xpcall'
    [@programdir\core\base\utils.lua:275]:
    [@programdir\core\tool\linker.lua:221]: in function 'link'
    [@programdir\actions\build\kinds\static.lua:53]: in function 'callback'
    [@programdir\modules\core\project\depend.lua:217]: in function 'on_changed'
    [@programdir\actions\build\kinds\static.lua:41]: in function '_do_link_target'
    [@programdir\actions\build\kinds\static.lua:84]:
    [@programdir\actions\build\kinds\static.lua:111]: in function '_link_target'
    [@programdir\actions\build\kinds\static.lua:139]: in function 'jobfunc'
    [@programdir\modules\async\runjobs.lua:241]:
    [C]: in function 'xpcall'
    [@programdir\core\base\utils.lua:275]: in function 'trycall'
    [@programdir\core\sandbox\modules\try.lua:117]: in function 'try'
    [@programdir\modules\async\runjobs.lua:223]: in function 'cotask'
    [@programdir\core\base\scheduler.lua:406]:

stack traceback:
        [C]: in function 'error'
        @programdir\core\base\os.lua:973: in function 'base/os.raiselevel'
        (...tail calls...)
        @programdir\core\main.lua:329: in upvalue 'cotask'
        @programdir\core\base\scheduler.lua:406: in function <@programdir\core\base\scheduler.lua:399>
@iDingDong iDingDong added the bug label Jul 18, 2024
@star-hengxing
Copy link
Contributor

Please use set_kind("moduleonly")

@waruqi
Copy link
Member

waruqi commented Jul 18, 2024

如果没有 .cpp 只要纯 module 文件,得用 moduleonly 而不是 static

@iDingDong
Copy link
Author

Please use set_kind("moduleonly")

That would not produce a static library for me. Is it impossible to build a static library that exports modules?

@waruqi
Copy link
Member

waruqi commented Jul 19, 2024

see #4707 (comment)

@iDingDong
Copy link
Author

see #4707 (comment)

This comment doesn't make sense. Module units do get built into .lib at least for MSVC.

@iDingDong
Copy link
Author

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
	int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};

@waruqi
Copy link
Member

waruqi commented Jul 19, 2024

@Arthapz

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


@arthapaj

@Arthapz
Copy link
Member

Arthapz commented Jul 19, 2024

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
	int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};

and it work (but i get a 4KB static library :D)

image
image

@Arthapz
Copy link
Member

Arthapz commented Jul 19, 2024

as for your first example, i think the module get culled as it import and export nothing, we should add a proper error

but if ur static library only contains named module, u should use moduleonly targetkind to avoid flag compatibility issues with the consumer

@iDingDong
Copy link
Author

iDingDong commented Jul 19, 2024

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
	int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};

and it work (but i get a 4KB static library :D)

Size doesn't matter as long as it is reasonably larger than an empty library. It could be affected by other configurations.

Latest xmake would compile it but failed to resolve dependencies when another target attempt to add the library as its dependency and imports the module.

as for your first example, i think the module get culled as it import and export nothing, we should add a proper error

The initial example serves as a minimal repro case which does not really need to define another symbol to make itself clear.

I do appreciate a warning but an empty lib is prefered than an error.

but if ur static library only contains named module, u should use moduleonly targetkind to avoid flag compatibility issues with the consumer

That kind of problem is not exclusive to modules that any static library built out of old-fashion source/header can be troubled the same way. Since it has never been a legit reason to drop static library support for tranditional sources and switch, modules should be nothing different.

@Arthapz
Copy link
Member

Arthapz commented Jul 21, 2024

u need to flag ur named module as public to get them accessible from other targets and not being culled

target("testm")
  set_kind("static")
  set_languages("cxxlatest")
  add_files("./test.cppm", {public = true})
target_end()

by default we cull non-public unreferenced named module (a named module which is netheir imported inside the library or exposed to be consumed) to avoid building unused modules

    local objectfiles_sorted_set = hashset.from(objectfiles_sorted)
    for _, objectfile in ipairs(objectfiles) do
        if not objectfiles_sorted_set:has(objectfile) then
            -- cull unreferenced non-public named module but add non-module files and implementation modules
            local _, provide, cppfile = compiler_support.get_provided_module(modules[objectfile])
            local fileconfig = target:fileconfig(cppfile)
            local public = fileconfig and fileconfig.public
            if not provide or public then
                table.insert(result, objectfile)
            end
        end
    end
    return result

for _, objectfile in ipairs(objectfiles) do

@Arthapz
Copy link
Member

Arthapz commented Jul 21, 2024

#5369 i also added a policy to disable project or target wise module culling
and a fileconfig entry to disable culling on a file

add_rules("mode.release", "mode.debug")
set_languages("c++20")

-- project wise
-- set_policy("build.c++.modules.culling", false)

target("culling")
    set_kind("static")
    add_files("src/*.mpp")
    -- target wise
    set_policy("build.c++.modules.culling", false)
target("culling")
    set_kind("static")
    add_files("src/*.mpp", {cull = false}) -- on files

this should provide to the user the flexibility to choose what he want, but i don't think it's a good default so please use public visibility in first attempt

and i added a proper warning for culled modules
image

@waruqi
Copy link
Member

waruqi commented Jul 25, 2024

Does it work now? try dev again. xmake update -s dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants