Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update gbl to V03-01-01 and millepede to V04-16-00 #9053

Merged
merged 2 commits into from
Mar 9, 2024

Conversation

sroychow
Copy link
Contributor

@sroychow sroychow commented Mar 6, 2024

  • Update of GBL software version for tracker alignment calibration with a fix for logic reported in comment
  • Update of Millepede II version for Tracker Alignment

Backport to 14.0 will also be needed. Supersedes #8976

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 6, 2024

A new Pull Request was created by @sroychow for branch IB/CMSSW_14_1_X/master.

@iarspider, @smuzaffar, @aandvalenzuela, @cmsbuild can you please review it and eventually sign? Thanks.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 6, 2024

cms-bot internal usage

@mmusich
Copy link

mmusich commented Mar 6, 2024

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 6, 2024

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37940/summary.html
COMMIT: fca3ab7
CMSSW: CMSSW_14_1_X_2024-03-06-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37940/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

make[2]: *** [CMakeFiles/GBL.dir/build.make:135: CMakeFiles/GBL.dir/src/GblTrajectory.cpp.o] Error 1
make[2]: Leaving directory '/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/gbl/V03-01-01-6f06aefc0167fb2cc2f3b2e331d7ba48/gbl-V03-01-01/build'
make[1]: *** [CMakeFiles/Makefile2:888: CMakeFiles/GBL.dir/all] Error 2
make[1]: Leaving directory '/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/gbl/V03-01-01-6f06aefc0167fb2cc2f3b2e331d7ba48/gbl-V03-01-01/build'
make: *** [Makefile:149: all] Error 2
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.tm6lIO (%build)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+gbl+V03-01-01-6f06aefc0167fb2cc2f3b2e331d7ba48
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.tm6lIO (%build)


@iarspider
Copy link
Contributor

@mmusich @sroychow I think the problem is that gbl is forcing C++ 11 here: https://gitlab.desy.de/claus.kleinwort/general-broken-lines/-/blob/V03-01-01/cpp/CMakeLists.txt?ref_type=tags#L32, while Eigen requires at least C++ 14 (and we build Eigen with C++ 17).

@mmusich
Copy link

mmusich commented Mar 7, 2024

@iarspider

I think the problem is that gbl is forcing C++ 11 here

thanks. The GBL developer confirmed that the ‘CMakeLists.txt’ supplied with GBL is only a default version. It should be customised by the user, e.g. for the c++ standard:

# set CMAKE_CXX_STANDARD
IF( NOT ${CMAKE_VERSION} VERSION_LESS "3.1.0" )
  IF( SUPPORT_ROOT )
    SET( CMAKE_CXX_STANDARD 17 )
  ELSE()
    SET( CMAKE_CXX_STANDARD 11 )
  ENDIF()
ELSE()
# defined in cmake/ilcsoft_default_cxx_flags.cmake
ENDIF()

The "SET( CMAKE_CXX_STANDARD 11 )” could be replaced with a "SET( CMAKE_CXX_STANDARD 17 )".
What is the @cms-sw/externals-l2 recommendation to proceed? To make a clone of the original repo and make the cms-sw externals fetch from there, or is there a way to inject the change in the CMake as top level replacement?
If the former is an option, do you have recommendations on where such a "private" fork should live?
Thank you.

@iarspider
Copy link
Contributor

iarspider commented Mar 7, 2024

@mmusich the following patch (on top of proposed changes) should work:

-- a/gbl.spec
+++ b/gbl.spec
@@ -8,6 +8,8 @@ Requires: eigen
 
 %prep
 %setup -q -n %{n}-%{realversion}
+grep -q 'CMAKE_CXX_STANDARD  *11' cpp/CMakeLists.txt
+sed -i -e 's|CMAKE_CXX_STANDARD  *11|CMAKE_CXX_STANDARD %{cms_cxx_standard}|' cpp/CMakeLists.txt
 
 %build
 rm -rf build
@@ -21,11 +23,10 @@ cmake ../cpp \
   -DEIGEN3_INCLUDE_DIR=${EIGEN_ROOT}/include/eigen3 \
   -DSUPPORT_ROOT=False \
   %ifarch x86_64
-  -DCMAKE_CXX_FLAGS="-DEIGEN_MAX_ALIGN_BYTES=64 -msse3" \
+  -DCMAKE_CXX_FLAGS="-DEIGEN_MAX_ALIGN_BYTES=64 -msse3"
   %else
-  -DCMAKE_CXX_FLAGS="-DEIGEN_MAX_ALIGN_BYTES=64" \
+  -DCMAKE_CXX_FLAGS="-DEIGEN_MAX_ALIGN_BYTES=64"
   %endif
-  -DCMAKE_CXX_STANDARD=%{cms_cxx_standard}

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2024

Pull request #9053 was updated.

@sroychow
Copy link
Contributor Author

sroychow commented Mar 7, 2024

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2024

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37956/summary.html
COMMIT: 522dd1f
CMSSW: CMSSW_14_1_X_2024-03-06-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37956/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

>> Compiling  src/Alignment/ReferenceTrajectories/src/TrajectoryFactoryBase.cc
>> Compiling  src/Alignment/ReferenceTrajectories/src/TrajectoryFactoryPlugin.cc
>> Compiling  src/Alignment/ReferenceTrajectories/src/TwoBodyDecayTrajectory.cc
>> Compiling  src/Alignment/ReferenceTrajectories/src/TwoBodyDecayTrajectoryState.cc
src/Alignment/ReferenceTrajectories/src/ReferenceTrajectory.cc: In member function 'virtual bool ReferenceTrajectory::addMaterialEffectsLocalGbl(const std::vector&, const std::vector&, const std::vector&, const std::vector&)':
src/Alignment/ReferenceTrajectories/src/ReferenceTrajectory.cc:1027:29: error: call of overloaded 'addScatterer(Eigen::CwiseNullaryOp, Eigen::Matrix >&, const Eigen::Inverse >)' is ambiguous
 1027 |       aGblPoint.addScatterer(scatterer, scatPrecision.inverse());
      |       ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /pool/condor/dir_3756739/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc12/external/gbl/V03-01-01-0bb1f00d4c29125130ee01bad4382681/include/GblTrajectory.h:36,
                 from /pool/condor/dir_3756739/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-03-06-2300/src/Alignment/ReferenceTrajectories/interface/ReferenceTrajectoryBase.h:103,
                 from /pool/condor/dir_3756739/jenkins/workspace/ib-run-pr-tests/CMSSW_14_1_X_2024-03-06-2300/src/Alignment/ReferenceTrajectories/interface/ReferenceTrajectory.h:38,


@mmusich
Copy link

mmusich commented Mar 7, 2024

please test with cms-sw/cmssw#44340

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 8, 2024

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37959/summary.html
COMMIT: 522dd1f
CMSSW: CMSSW_14_1_X_2024-03-06-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37959/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37959/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testPhase2PixelNtuple had ERRORS

RelVals

----- Begin Fatal Exception 08-Mar-2024 08:46:37 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_DIGI_L1TrackTrigger_L1_DIGI2RAW_HLT.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/VertexFinder/data/NNVtx_WeightModelGraph.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/24896.0_CloseByPGun_CE_E_Front_120um+2026D98


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(887): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(972): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(965): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(519): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1489): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Mar-2024 08:46:42 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_DIGI_L1TrackTrigger_L1_DIGI2RAW_HLT.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/VertexFinder/data/NNVtx_WeightModelGraph.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/24900.0_CloseByPGun_CE_H_Coarse_Scint+2026D98


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(887): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(972): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(965): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(519): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1489): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Mar-2024 08:50:09 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_DIGI_L1TrackTrigger_L1_DIGI2RAW_HLT.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/VertexFinder/data/NNVtx_WeightModelGraph.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-06-2300/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/24834.0_TTbar_14TeV+2026D98


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(887): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(972): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Types.py(965): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(384): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Mixins.py(519): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37959/CMSSW_14_1_X_2024-03-06-2300/src/FWCore/ParameterSet/python/Config.py(1489): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 20834.020834.0_TTbar_14TeV+2026D88/step2_TTbar_14TeV+2026D88.log
  • 23634.023634.0_TTbar_14TeV+2026D95/step2_TTbar_14TeV+2026D95.log
  • 24034.024034.0_TTbar_14TeV+2026D96/step2_TTbar_14TeV+2026D96.log
Expand to see more relval errors ...

@mmusich
Copy link

mmusich commented Mar 8, 2024

Failures are unrelated, let's try again

@mmusich
Copy link

mmusich commented Mar 8, 2024

test parameters:

@mmusich
Copy link

mmusich commented Mar 8, 2024

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 8, 2024

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37984/summary.html
COMMIT: 522dd1f
CMSSW: CMSSW_14_1_X_2024-03-08-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/9053/37984/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37984/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-968716/37984/git-merge-result

Comparison Summary

Summary:

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 9, 2024

REMINDER @antoniovilela, @sextonkennedy, @rappoccio: This PR was tested with cms-sw/cmssw#44340, please check if they should be merged together

@perrotta
Copy link
Contributor

perrotta commented Mar 9, 2024

+alca

  • (FWIW)

@antoniovilela
Copy link

+1

@antoniovilela
Copy link

merge

  • I understand this can be merged along with the cmssw partner, and a second cmsdist update will come independently later.

@cmsbuild cmsbuild merged commit 8edd672 into cms-sw:IB/CMSSW_14_1_X/master Mar 9, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants