Update Millepede to V04-14-00 #8953

consuegs · 2024-01-18T13:32:03Z

Update of Millepede software version for tracker alignment calibration
Introduction of Intel oneMKL PARDISO solution method. PARDISO is to become from now on the solution method of choice for (special) problems with really sparse global matrices
See presentation here at TrkAl meeting (09.01.2024) by C. Kleinwort with overview of method and performance summary from test campaign

cmsbuild · 2024-01-18T13:32:27Z

A new Pull Request was created by @consuegs for branch IB/CMSSW_14_0_X/master.

@cmsbuild, @aandvalenzuela, @smuzaffar, @iarspider can you please review it and eventually sign? Thanks.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2024-01-18T13:32:27Z

cms-bot internal usage

smuzaffar · 2024-01-18T13:33:18Z

cmsbuild, please test

cmsbuild · 2024-01-18T16:52:11Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b707f3/36919/summary.html
COMMIT: e38051d
CMSSW: CMSSW_14_0_X_2024-01-18-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8953/36919/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 98 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3247526
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3247498
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: no differences found

smuzaffar · 2024-01-18T21:06:55Z

+externals

smuzaffar · 2024-01-18T21:07:14Z

assign alca

@cms-sw/alca-l2 , can you please review this?

cmsbuild · 2024-01-18T21:07:24Z

New categories assigned: alca

@saumyaphor4252,@perrotta,@consuegs you have been requested to review this Pull request/Issue and eventually sign? Thanks

mmusich · 2024-01-19T13:45:36Z

I am a bit puzzled by some of the unit tests outputs.
e.g.:

testPedeCampaign seems to have run correctly, but then the last mv fails (due to permission denied). In IBs as far as I can see instead there is no such issue.
testPayloadSanity consequently fails due to missing input, but despite it should track the runtime cmsRun failure [1], the test is still declared as succeeded.

[1] https://github.com/cms-sw/cmssw/blob/87aa86684413a8056be46ed8bc1377e4c9954d66/Alignment/MillePedeAlignmentAlgorithm/test/test_payload_sanity.sh#L5

mmusich · 2024-01-19T13:59:57Z

consequently fails due to missing input, but despite it should track the runtime cmsRun failure [1], the test is still declared as succeeded.

OK, this is due to a bug in the script, see fix at cms-sw/cmssw#43754.

mmusich · 2024-01-19T14:00:48Z

test parameters:

workflows = 1001.0, 1001.2, 1001.3
pull_request = fix test_payload_sanity unit test script cmssw#43754

mmusich · 2024-01-19T14:02:14Z

@cmsbuild, please test

cmsbuild · 2024-01-19T17:39:47Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b707f3/36936/summary.html
COMMIT: e38051d
CMSSW: CMSSW_14_0_X_2024-01-19-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8953/36936/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test testPayloadSanity had ERRORS

Comparison Summary

Summary:

You potentially removed 5 lines from the logs
Reco comparison results: 19 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3262461
DQMHistoTests: Total failures: 1212
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3261227
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 211 log files, 163 edm output root files, 50 DQM output files
TriggerResults: no differences found

mmusich · 2024-01-19T18:25:54Z

@cmsbuild, please test

now it should succeed

cmsbuild · 2024-01-20T00:17:53Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b707f3/36943/summary.html
COMMIT: e38051d
CMSSW: CMSSW_14_0_X_2024-01-19-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8953/36943/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@bsunanda Phase2-hgx350C Steps to create a scenario of B18 Geometry of HGCal where no innecr cell straucture of full wafers are not created atin Geant4 geometry cmssw#43718
@smuzaffar Unit tests to check SCRAM test command works cmssw#43719
@silviodonato Add fillDescription to CommonTools/UtilAlgos/interface/Merger.h - 140X cmssw#43731
@lathomas TeV jet RAW-RECO skim cmssw#43581
@Dr15Jones Remove beginRun/beginLuminosityBlock methods in EGammaElectronProducers cmssw#43665
@slava77 add OT tracker cluster masks in TrackingNtuple cmssw#43720
@mmusich Adapt shortened tracks resolution method to work with AlCaReco event content cmssw#43729
@Dr15Jones Handle empty branch index lists from input cmssw#43747
@makortel Example of Alpaka-based EventSetup data product that uses PortableCollection member variables cmssw#43682
@mmusich Update testTrackAnalyzers cmssw#43716
@bsunanda Phase2-hgx351 Modify to analyze flat files contatining only full silisonlayers of EE and HE (usefule for HFNose) cmssw#43717
@cgsavard Update L1 track jet cuts cmssw#43610
@etzia Add Run3 promptData JetID cmssw#43195
@mitaylor Add caloParams v0_4_2 and v0_4_3 cmssw#43046
@arsahasransu Electron and Photon Scouting Data Format Update for 2024 pp Collisions cmssw#43744
@thesps L1T Seeded Cone jets update cmssw#43233
@cms-trackeralign ReferenceTrajectory: decrease the value of the minumum precision to allow along-strip alignment of Phase2 2S modules cmssw#43745
@Dr15Jones DQMStore avoids using 0 as invalid module ID cmssw#43724
@cms-sw Update tag for DataFormats-L1ScoutingRawData to V00-01-00 #8954
@cms-sw Update tag for DataFormats-L1Scouting to V00-01-00 #8955

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b707f3/36943/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b707f3/36943/git-merge-result

Comparison Summary

Summary:

You potentially removed 3 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3262299
DQMHistoTests: Total failures: 2110
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3260167
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 37.96199999999999 KiB( 49 files compared)
DQMHistoSizes: changed ( 1000.0,... ): 1.406 KiB Tracking/ShortTrackResolution
Checked 211 log files, 163 edm output root files, 50 DQM output files
TriggerResults: no differences found

perrotta · 2024-01-21T08:39:42Z

+1

Thank you @mmusich for the fixes in the unit tests

cmsbuild · 2024-01-21T08:40:07Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_14_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/cmssw#43754

smuzaffar · 2024-01-23T10:20:12Z

As cms-sw/cmssw#43754 has been merged so this is good to go in 14.0.X now

Update Millepede to V04-14-00

e38051d

cmsbuild added tests-pending externals-pending pending-signatures orp-pending labels Jan 18, 2024

cmsbuild added tests-started and removed tests-pending labels Jan 18, 2024

cmsbuild added tests-approved and removed tests-started labels Jan 18, 2024

cmsbuild added alca-pending externals-approved and removed externals-pending labels Jan 18, 2024

mmusich mentioned this pull request Jan 19, 2024

fix test_payload_sanity unit test script cms-sw/cmssw#43754

Merged

cmsbuild added the requires-external label Jan 19, 2024

cmsbuild added tests-started and removed tests-approved labels Jan 19, 2024

cmsbuild added tests-rejected and removed tests-started labels Jan 19, 2024

cmsbuild added tests-started and removed tests-rejected labels Jan 19, 2024

cmsbuild added tests-approved and removed tests-started labels Jan 20, 2024

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jan 21, 2024

smuzaffar merged commit 51c2c65 into cms-sw:IB/CMSSW_14_0_X/master Jan 23, 2024
11 checks passed

This was referenced Jan 23, 2024

Pythia 8.311 #8938

Merged

fix gsl hook to use openblas libs cms-sw/cms-common#10

Merged

cms-common: fixed gsl runtime hook to include openblas libs #8959

Merged

Fix gsl-config so that by default it uses openblas lib #8958

Merged

henriettepetersen mentioned this pull request Jan 30, 2024

Update GBL to V03-01-00 #8976

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Millepede to V04-14-00 #8953

Update Millepede to V04-14-00 #8953

consuegs commented Jan 18, 2024 •

edited

Loading

cmsbuild commented Jan 18, 2024

cmsbuild commented Jan 18, 2024 •

edited

Loading

smuzaffar commented Jan 18, 2024

cmsbuild commented Jan 18, 2024

smuzaffar commented Jan 18, 2024

smuzaffar commented Jan 18, 2024

cmsbuild commented Jan 18, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

cmsbuild commented Jan 19, 2024

mmusich commented Jan 19, 2024

cmsbuild commented Jan 20, 2024

perrotta commented Jan 21, 2024

cmsbuild commented Jan 21, 2024

smuzaffar commented Jan 23, 2024

Update Millepede to V04-14-00 #8953

Update Millepede to V04-14-00 #8953

Conversation

consuegs commented Jan 18, 2024 • edited Loading

cmsbuild commented Jan 18, 2024

cmsbuild commented Jan 18, 2024 • edited Loading

smuzaffar commented Jan 18, 2024

cmsbuild commented Jan 18, 2024

Comparison Summary

smuzaffar commented Jan 18, 2024

smuzaffar commented Jan 18, 2024

cmsbuild commented Jan 18, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

mmusich commented Jan 19, 2024

cmsbuild commented Jan 19, 2024

Unit Tests

Comparison Summary

mmusich commented Jan 19, 2024

cmsbuild commented Jan 20, 2024

Comparison Summary

perrotta commented Jan 21, 2024

cmsbuild commented Jan 21, 2024

smuzaffar commented Jan 23, 2024

consuegs commented Jan 18, 2024 •

edited

Loading

cmsbuild commented Jan 18, 2024 •

edited

Loading