Update TBB to 2021.2.0 and use cmake to build #6792

mrodozov · 2021-04-07T12:12:59Z

this is using existing PR + additional flag and new version

cmsbuild · 2021-04-07T12:13:21Z

A new Pull Request was created by @mrodozov (Mircho Rodozov) for branch IB/CMSSW_11_3_X/master.

@cmsbuild, @smuzaffar, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

mrodozov · 2021-04-07T12:15:14Z

please test

mrodozov · 2021-04-07T12:21:25Z

@fwyzard do we need this patch and what it does (I didn't check what is actually doing), and also can I remove the -DCMAKE_BUILD_TYPE=RelWithDebInfo because it's a debug build or you want to check what it's doing first ?

cmsbuild · 2021-04-07T16:10:12Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14066/summary.html
COMMIT: e555977
CMSSW: CMSSW_11_3_X_2021-04-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6792/14066/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

[BuildRules] updates Intel micro-arch maps for skylakeavx512 builds #6791 @cms-sw: [BuildRules] updates Intel micro-arch maps for skylakeavx512 builds

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14066/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14066/git-merge-result

Build

I found compilation error when building:

>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/SimG4Core/Physics/src/PhysicsList.cc
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/SimG4Core/Physics/src/PhysicsListFactory.cc
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/FWCore/PluginManager/interface/PluginFactory.h:27,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/SimG4Core/Physics/interface/PhysicsListFactory.h:7,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/SimG4Core/Physics/src/PhysicsListFactory.cc:1:
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/FWCore/PluginManager/interface/PluginFactoryBase.h:64:58: error: 'zero_allocator' is not a member of 'tbb'
   64 |     typedef tbb::concurrent_vector> PMakers;
      |                                                          ^~~~~~~~~~~~~~
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_3_X_2021-04-06-2300/src/FWCore/PluginManager/interface/PluginFactoryBase.h:64:73: error: template argument 2 is invalid
   64 |     typedef tbb::concurrent_vector> PMakers;
      |                                                                         ^~~~~~~~~~~~~~~

makortel · 2021-04-07T18:30:12Z

.../FWCore/PluginManager/interface/PluginFactoryBase.h:64:58: error: 'zero_allocator' is not a member of 'tbb'

I see the zero_allocator has indeed been removed from oneTBB, but I wasn't able to find any information on why. @Dr15Jones

Dr15Jones · 2021-04-07T18:45:07Z

The oneAPI documentation for std::concurrent_vector says "You must synchronize construction and access." Maybe they changed their policy and put a much greater burden on the users?

Dr15Jones · 2021-04-07T20:03:36Z

@dan131riley thoughts?

dan131riley · 2021-04-07T21:00:21Z

The oneAPI documentation for std::concurrent_vector says "You must synchronize construction and access." Maybe they changed their policy and put a much greater burden on the users?

That language was in the docs before oneAPI. Also strange, zero_allocator is still in the oneAPI spec as is the advice to use it for waiting on an element.

I think the zero allocator is still the right thing to do there. It isn't a complicated class, so we could just copy it into FWCore/Utilities and s/tbb/edm/.

I also opened an issue.

fwyzard · 2021-04-07T21:38:37Z

@mrodozov

@fwyzard do we need this patch and what it does (I didn't check what is actually doing)

If you mean the changes related to tbbbind and hwloc, yes, please keep them.

If you mean the part about

+# Patch tbb v2021.1.1 CMake file to support $HWLOC_ROOT
+# not needed in the tbb master branch
+Patch: tbb-2021.1.1-cmake_policy-CMP0074

then I think it can be dropped: according to the documentation we can use the same approach as in #6525 .

can I remove the -DCMAKE_BUILD_TYPE=RelWithDebInfo because it's a debug build or you want to check what it's doing first ?

I actually think we should keep RelWithDebInfo - as far as I know it just means that TBB will be built with the -g flag, so we still get an optimised build, but we also get debug symbols.

fwyzard

Update the spec file to use the syntax for using the hwloc library introduced in TBB v2021.2.0.

tbb.spec

cmsbuild · 2021-04-08T09:49:59Z

Pull request #6792 was updated.

mrodozov · 2021-04-08T09:52:38Z

please test
checking the ext build

gartung · 2021-04-22T19:55:36Z

please test with cms-sw/cmssw#33474

cmsbuild · 2021-04-23T08:05:03Z

-1

Failed Tests: HeaderConsistency
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14531/summary.html
COMMIT: 0b05726
CMSSW: CMSSW_12_0_X_2021-04-22-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6792/14531/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Rivet 3.1.4, YODA 1.9.0, Professor 2.3.2 #6822 @mseidel42: Rivet 3.1.4, YODA 1.9.0, Professor 2.3.2
Coral - Fix for limiting idle connection pool size #6830 @cms-sw: Coral - Fix for limiting idle connection pool size

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14531/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14531/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2877046
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2877023
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

gartung · 2021-04-23T14:19:25Z

please test with cms-sw/cmssw#33474

cmsbuild · 2021-04-24T01:19:21Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14544/summary.html
COMMIT: 0b05726
CMSSW: CMSSW_12_0_X_2021-04-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/6792/14544/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

move crab-dev to v3.210422 #6832 @ddaina: move crab-dev to v3.210422

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14544/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14544/git-merge-result

Found compilation warnings

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2877046
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2877011
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 37 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

makortel · 2021-04-30T15:11:36Z

please test with cms-sw/cmssw#33474

makortel · 2021-04-30T15:12:05Z

Is there a difference between starting tests from cmssw vs cvmsdist?

mrodozov · 2021-04-30T16:46:30Z

the 'tests approved' label will appear only on the PR from where you are testing
before it did on both places but that was not optimal - for example you might have wanted to test the cmssw alone, or with the cmsdist + other PRs.
the test labels will appear only on the PR domain from which they were started
but the testing alone - no. there is no difference

makortel · 2021-04-30T18:01:12Z

but the testing alone - no. there is no difference

Ok, thanks. The last test launched from cmssw side just failed in a way that looked suspicious (like TBB library would be missing).

cmsbuild · 2021-04-30T22:58:53Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14757/summary.html
COMMIT: 0b05726
CMSSW: CMSSW_12_0_X_2021-04-29-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/6792/14757/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Keep the PTX in CUDA binaries, so it can be JIT'ted for newer devices #6851 @fwyzard: Keep the PTX in CUDA binaries, so it can be JIT'ted for newer devices

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14757/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ca273c/14757/git-merge-result

Found compilation warnings

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2662646
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2662623
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: no differences found

silviodonato · 2021-05-04T16:33:20Z

+1

Update TBB to 2021.2.0 and use cmake to build

e555977

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Apr 7, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 7, 2021

cmsbuild added tests-rejected and removed tests-started labels Apr 7, 2021

This was referenced Apr 7, 2021

Update TBB to the master branch #6525

Closed

Update TBB to version v2021.1.1 #6544

Closed

fwyzard suggested changes Apr 7, 2021

View reviewed changes

tbb.spec Outdated Show resolved Hide resolved

tbb.spec Outdated Show resolved Hide resolved

tbb.spec Outdated Show resolved Hide resolved

remove patch and add a flag

0b05726

cmsbuild added tests-pending and removed tests-rejected labels Apr 8, 2021

cmsbuild added tests-started tests-rejected and removed tests-pending tests-started labels Apr 8, 2021

cmsbuild added tests-started and removed tests-approved labels Apr 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Apr 23, 2021

cmsbuild added tests-started and removed tests-rejected labels Apr 23, 2021

cmsbuild added tests-approved and removed tests-started labels Apr 24, 2021

cmsbuild added tests-started and removed tests-approved labels Apr 30, 2021

cmsbuild added tests-approved and removed tests-started labels Apr 30, 2021

cmsbuild added orp-approved and removed orp-pending labels May 4, 2021

cmsbuild merged commit b982e6f into cms-sw:IB/CMSSW_12_0_X/master May 4, 2021

This was referenced May 4, 2021

Add libunwind dependency on zlib. #6873

Merged

drop high-level python2 tools #6848

Merged

Removal of many more python2 tools #6869

Closed

dan131riley mentioned this pull request May 5, 2021

segfaults in ThreadSafeLogMessageLoggerScribe on slc7_ppc64le_gcc9 cms-sw/cmssw#33636

Open

mweisgut mentioned this pull request Oct 29, 2021

Status of tbb::zero_allocator? oneapi-src/oneTBB#378

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update TBB to 2021.2.0 and use cmake to build #6792

Update TBB to 2021.2.0 and use cmake to build #6792

mrodozov commented Apr 7, 2021

cmsbuild commented Apr 7, 2021

mrodozov commented Apr 7, 2021

mrodozov commented Apr 7, 2021 •

edited

Loading

cmsbuild commented Apr 7, 2021

makortel commented Apr 7, 2021

Dr15Jones commented Apr 7, 2021

Dr15Jones commented Apr 7, 2021

dan131riley commented Apr 7, 2021

fwyzard commented Apr 7, 2021 •

edited

Loading

fwyzard left a comment

cmsbuild commented Apr 8, 2021

mrodozov commented Apr 8, 2021

gartung commented Apr 22, 2021

cmsbuild commented Apr 23, 2021

gartung commented Apr 23, 2021

cmsbuild commented Apr 24, 2021

makortel commented Apr 30, 2021 •

edited

Loading

makortel commented Apr 30, 2021

mrodozov commented Apr 30, 2021 •

edited

Loading

makortel commented Apr 30, 2021

cmsbuild commented Apr 30, 2021

silviodonato commented May 4, 2021

Update TBB to 2021.2.0 and use cmake to build #6792

Update TBB to 2021.2.0 and use cmake to build #6792

Conversation

mrodozov commented Apr 7, 2021

cmsbuild commented Apr 7, 2021

mrodozov commented Apr 7, 2021

mrodozov commented Apr 7, 2021 • edited Loading

cmsbuild commented Apr 7, 2021

Build

makortel commented Apr 7, 2021

Dr15Jones commented Apr 7, 2021

Dr15Jones commented Apr 7, 2021

dan131riley commented Apr 7, 2021

fwyzard commented Apr 7, 2021 • edited Loading

fwyzard left a comment

Choose a reason for hiding this comment

cmsbuild commented Apr 8, 2021

mrodozov commented Apr 8, 2021

gartung commented Apr 22, 2021

cmsbuild commented Apr 23, 2021

Comparison Summary

gartung commented Apr 23, 2021

cmsbuild commented Apr 24, 2021

Comparison Summary

makortel commented Apr 30, 2021 • edited Loading

makortel commented Apr 30, 2021

mrodozov commented Apr 30, 2021 • edited Loading

makortel commented Apr 30, 2021

cmsbuild commented Apr 30, 2021

Comparison Summary

silviodonato commented May 4, 2021

mrodozov commented Apr 7, 2021 •

edited

Loading

fwyzard commented Apr 7, 2021 •

edited

Loading

makortel commented Apr 30, 2021 •

edited

Loading

mrodozov commented Apr 30, 2021 •

edited

Loading