Add a test for the dependency on host-only types from alpaka libraries

[fwyzard]

PR description:

Add a test for the dependency on host-only types from alpaka libraries.

Based on the original example by @ariostas.

PR validation:

The new unit test is expected to pass, but fails because the build system cannot habdle this kind of dependency, yet.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Could be backported to 14.0.x once the build rules are updated.

[cmsbuild]

cms-bot internal usage

[fwyzard]

@ariostas thank you for providing the original reproducer.

I've discussed how to fix the dependencies with @smuzaffar, and we think the best approach will be to introduce a new flag to the BuikdFile, e.g. <flags ALPAKA_LINK="1"/>.

Once that is available, I will update the BuildFile of this test.

[fwyzard]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45285/40679

• This PR adds an extra 20KB to repository

[cmsbuild]

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Pull Request was created by @fwyzard for master.

It involves the following packages:

• HeterogeneousCore/AlpakaTest (heterogeneous)

@makortel, @Dr15Jones, @smuzaffar, @fwyzard can you please review it and eventually sign? Thanks.
@rovere, @missirol, @makortel this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa4396/40023/summary.html
COMMIT: f285ad2
CMSSW: CMSSW_14_1_X_2024-06-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45285/40023/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH='CMSSW_14_1_X_2024-06-22-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_14_1_X_2024-06-22-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/alpaka/1.1.0-0a6641b4bfdf883c8da5b9e8620be504/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/pcre/8.43-e34796d17981e9b6d174328c69446455/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/boost/1.80.0-a1544032d9d65904ac2112b6d35bba55/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/lcg/root/6.30.07-5b5d83179d6ed7bc60aa7a5006899eba/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/tbb/v2021.9.0-1dc6d66b4b014f4ae4733b04914ce7d4/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/zlib/1.2.11-1a082fc322b0051b504cc023f21df178/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/fmt/8.0.1-258b4791803c34b7e98cf43693e54d87/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/tinyxml2/6.2.0-88fe0ec301baf763fa3c485e5b67ed91/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DALPAKA_DEFAULT_HOST_MEMORY_ALIGNMENT=128 -DALPAKA_DISABLE_VENDOR_RNG -DALPAKA_ACC_GPU_CUDA_ENABLED -DALPAKA_ACC_GPU_CUDA_ONLY_MODE -DALPAKA_HOST_ONLY -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/printAnswer.cc.d src/HeterogeneousCore/AlpakaTest/src/alpaka/printAnswer.cc -o tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/printAnswer.cc.o
>> Building alpaka/cuda shared library tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/libHeterogeneousCoreAlpakaTestCudaAsync.so
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DALPAKA_DEFAULT_HOST_MEMORY_ALIGNMENT=128 -DALPAKA_DISABLE_VENDOR_RNG -DALPAKA_ACC_GPU_CUDA_ENABLED -DALPAKA_ACC_GPU_CUDA_ONLY_MODE -DALPAKA_HOST_ONLY -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -shared -Wl,-E    -Wl,-z,defs     tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/ES_AlpakaESTestData.cc.o tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/printAnswer.cc.o -o tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/libHeterogeneousCoreAlpakaTestCudaAsync.so -Wl,-E -Wl,--hash-style=gnu -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-22-1100/biglib/el8_amd64_gcc12 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-22-1100/lib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/biglib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/lib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/external/el8_amd64_gcc12/lib -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-21-2300/lib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-21-2300/biglib/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/lib64/stubs -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-06-22-1100/static/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/static/el8_amd64_gcc12 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-06-21-2300/static/el8_amd64_gcc12 -lHeterogeneousCoreAlpakaCore -lHeterogeneousCoreAlpakaCoreCudaAsync -lFWCoreFramework -lFWCoreCommon -lFWCoreServiceRegistry -lDataFormatsCommon -lFWCoreParameterSet -lFWCoreMessageLogger -lDataFormatsProvenance -lFWCoreConcurrency -lFWCorePluginManager -lFWCoreReflection -lHeterogeneousCoreAlpakaInterface -lFWCoreUtilities -lFWCoreVersion -lTree -lNet -lThread -lMathCore -lRIO -lboost_program_options -lCore -lboost_thread -lboost_date_time -lpcre -lbz2 -lcudart -lcudadevrt -lnvToolsExt -luuid -ltbb -llzma -lz -lcuda -lfmt -lcms-md5 -lcrypt -ldl -lrt -lstdc++fs -ltinyxml2
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/../lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../x86_64-redhat-linux-gnu/bin/ld.bfd: tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/cc0hBLMD.ltrans0.ltrans.o: in function `alpaka_cuda_async::alpakatest::printAnswer()':
:(.text+0xb2): undefined reference to `alpakatest::HostOnlyType::print()'
collect2: error: ld returned 1 exit status
gmake: *** [tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestCudaAsync/libHeterogeneousCoreAlpakaTestCudaAsync.so] Error 1
Leaving library rule at src/HeterogeneousCore/AlpakaTest/src/alpaka
Entering library rule at src/HeterogeneousCore/AlpakaTest/src/alpaka
>> Compiling alpaka/rocm src/HeterogeneousCore/AlpakaTest/src/alpaka/ES_AlpakaESTestData.cc
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DGNU_GCC -D_GNU_SOURCE -D__HIP_PLATFORM_HCC__ -D__HIP_PLATFORM_AMD__ -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH='CMSSW_14_1_X_2024-06-22-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_14_1_X_2024-06-22-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-06-22-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/alpaka/1.1.0-0a6641b4bfdf883c8da5b9e8620be504/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/pcre/8.43-e34796d17981e9b6d174328c69446455/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/boost/1.80.0-a1544032d9d65904ac2112b6d35bba55/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/lcg/root/6.30.07-5b5d83179d6ed7bc60aa7a5006899eba/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/tbb/v2021.9.0-1dc6d66b4b014f4ae4733b04914ce7d4/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/zlib/1.2.11-1a082fc322b0051b504cc023f21df178/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/fmt/8.0.1-258b4791803c34b7e98cf43693e54d87/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/rocm/5.6.1-9b7217069df29cdf1c0bc74e165c09ec/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02842/el8_amd64_gcc12/external/tinyxml2/6.2.0-88fe0ec301baf763fa3c485e5b67ed91/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DALPAKA_DEFAULT_HOST_MEMORY_ALIGNMENT=128 -DALPAKA_DISABLE_VENDOR_RNG -DALPAKA_ACC_GPU_HIP_ENABLED -DALPAKA_ACC_GPU_HIP_ONLY_MODE -DALPAKA_HOST_ONLY -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestROCmAsync/ES_AlpakaESTestData.cc.d src/HeterogeneousCore/AlpakaTest/src/alpaka/ES_AlpakaESTestData.cc -o tmp/el8_amd64_gcc12/src/HeterogeneousCore/AlpakaTest/src/alpaka/HeterogeneousCoreAlpakaTestROCmAsync/ES_AlpakaESTestData.cc.o


[smuzaffar]

@ariostas thank you for providing the original reproducer.

I've discussed how to fix the dependencies with @smuzaffar, and we think the best approach will be to introduce a new flag to the BuikdFile, e.g. <flags ALPAKA_LINK="1"/>.

Once that is available, I will update the BuildFile of this test.

@fwyzard , I am implementing this and noticed that may be we can make use of existing USE_ALPAKA_<backend> flag. Currently build rules support e.g. USE_ALPAKA_CUDA to add additional deps for AlpakaCuda backend and USE_ALPAKA_ROCM to add additional deps for AlpakaRoc backend. How about I extent it to USE_ALPAKA which can provide addtional dependencies for all alpaka backends and one can either have <flags USE_ALPAKA="1"/> OR <flags USE_ALPAKA="Subsystem/Package"/> to add deps on its default library?

[smuzaffar]

cms-sw/cmsdist#9263 should allow to link against the default library by using <flags USE_ALPAKA="1"/> in the BuildFile.xml

[fwyzard]

So the BuildFile.xml would look like

<use name="FWCore/Framework"/>
<use name="HeterogeneousCore/AlpakaCore"/>
<use name="HeterogeneousCore/AlpakaInterface"/>
<flags ALPAKA_BACKENDS="1"/>
<flags USE_ALPAKA="1"/>
<export>
  <lib name="1"/>
</export>

?

It seems counter-intuitive to write USE_ALPAKA="1" to mean "the alpaka libraries depend on the non-alpaka library".

[smuzaffar]

@fwyzard, USE_ALPAKA keyword here was to indicate that it provides a dependency for alpaka backend products e.g. USE_ALPAKA_CUDA="depA" is same as if AlpakaCuda product had <use name="depA"/> and USE_ALPAKA_ROCM="depB" is same as if AlpakaRocm product had <use name="depB"/>. If it creates confusion then we can change the flag name

[smuzaffar]

One other way is to have src/Subsystem/Package/alpaka/BuildFile.xml with contents like

<use name="dep-for-all-backends"/>
<if backend="cuda">
  <use name="dep-for-cuda-backend"/>
</if>
<if backend="rocm">
  <use name="dep-for-rocm-backend"/>
</if>

but this need a lot more change both in build rules and scram but it is do able

[fwyzard]

One other way is to have src/Subsystem/Package/alpaka/BuildFile.xml with contents like

<use name="dep-for-all-backends"/>
<if backend="cuda">
  <use name="dep-for-cuda-backend"/>
</if>
<if backend="rocm">
  <use name="dep-for-rocm-backend"/>
</if>

but this need a lot more change both in build rules and scram but it is do able

Mhm, no, this looks a lot more complicated.

[fwyzard]

I just noticed that in HeterogeneousCore/AlpakaServices/BuildFile.xml we have

<!-- this dependency is needed only for the CUDA backend -->
<use name="HeterogeneousCore/CUDAServices" for="alpaka/cuda"/>

Would it work to add here

<!-- let the alpaka libraries depend on the main library -->
<use name="HeterogeneousCore/AlpakaTest" for="alpaka"/>

?

[fwyzard]

Currently this does work:

<!-- let the alpaka libraries depend on the main library -->
<use name="HeterogeneousCore/AlpakaTest" for="alpaka/cuda"/>
<use name="HeterogeneousCore/AlpakaTest" for="alpaka/rocm"/>
<use name="HeterogeneousCore/AlpakaTest" for="alpaka/serial"/>

[fwyzard]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa4396/40060/summary.html
COMMIT: 5b1d7b6
CMSSW: CMSSW_14_1_X_2024-06-24-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45285/40060/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 1 lines to the logs
• Reco comparison results: 5 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3345088
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3345062
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 202 log files, 165 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[fwyzard]

please test

To refresh the test results.

[cmsbuild]

-1

Failed Tests: RelVals
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa4396/40276/summary.html
COMMIT: 5b1d7b6
CMSSW: CMSSW_14_1_X_2024-07-07-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45285/40276/install.sh to create a dev area with all the needed externals and cmssw changes.

• DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals

• 29634.911A fatal system signal has occurred: segmentation violation

[fwyzard]

please test

[cmsbuild]

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa4396/40277/summary.html
COMMIT: 5b1d7b6
CMSSW: CMSSW_14_1_X_2024-07-07-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/45285/40277/install.sh to create a dev area with all the needed externals and cmssw changes.

• DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

There are some workflows for which there are errors in the baseline:
29634.911 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

• You potentially removed 21 lines from the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 15 differences found in the comparisons
• DQMHistoTests: Total files compared: 47
• DQMHistoTests: Total histograms compared: 3246793
• DQMHistoTests: Total failures: 847
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3245926
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 46 files compared)
• Checked 199 log files, 163 edm output root files, 47 DQM output files
• TriggerResults: no differences found

[makortel]

Sorry to jump in late (so maybe this would call for a separate issue), but would it be feasible to add the this dependency from the alpaka-backend libraries to the "main library" automatically for all Alpaka-enabled packages?

[fwyzard]

Hi @makortel I discussed this option with @smuzaffar.

The problem is that this will add a lot of extra dependencies to the alpaka libraries, including some dependencies on CUDA that we should not add to the non-CUDA backends.

It may be possible to improve this later, once

• we remove the native CUDA support from the framework;
• linking with --as-needed works (e.g. gcc: add --as-needed link option cmsdist#9232).

For now, adding this dependency only where it is explicitly needed seemed the safer option.

[makortel]

Ok. Maybe it would be worth of documenting these steps for possible eventual simplification in an issue?

[fwyzard]

I won't complain if you open an issue :-)

[makortel]

he problem is that this will add a lot of extra dependencies to the alpaka libraries, including some dependencies on CUDA that we should not add to the non-CUDA backends.

It may be possible to improve this later, once

• we remove the native CUDA support from the framework;
• linking with --as-needed works (e.g. gcc: add --as-needed link option cmsdist#9232).

Here #45285

I'd also suggest to (eventually) document these in https://github.com/cms-sw/cmssw/blob/master/HeterogeneousCore/AlpakaCore/README.md

[rappoccio]

+1