Migrate Phase-2 geometry to D98 #41946

srimanob · 2023-06-13T19:07:39Z

PR description:

This PR move Phase-2 baseline geometry to D98. I also use this PR to clean up old Phase-2 wfs from IB test.

PR validation:

Trying to dump the following workflows, and all seems to be fine:
24834.0,24834.911,24834.103,25061.97,24834.5,24834.9,25034.99,25034.999,24834.21,25034.21,25034.9921,25034.114,24896.0,24900.0

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

No need of backport

srimanob · 2023-06-13T19:09:39Z

Hi @mmusich @emiglior
Can 10% OT inefficiency worrkflow move to D98 (T32) from D88 (T24)?

cmsbuild · 2023-06-13T19:14:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41946/35918

This PR adds an extra 64KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Geometry/README.md modified in PR(s): Phase 2 tracker: realistic description of 3D modules in inner tracker #41880
- File Configuration/PyReleaseValidation/python/relval_2026.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255, Phase 2 tracker: realistic description of 3D modules in inner tracker #41880
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): More Configurable Pixel Tracks and HIon Template #41632, Add prototype HcalNanoAOD #41607, Enable feature to recycle GEN for FastSim #41945
- File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255, Updated nano matrix 13.2 #41867

cmsbuild · 2023-06-13T19:15:13Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/Geometry (geometry, upgrade)
Configuration/PyReleaseValidation (pdmv, upgrade)

@civanch, @Dr15Jones, @bsunanda, @makortel, @bbilin, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks.
@vargasa, @makortel, @kpedro88, @Martin-Grunewald, @missirol, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

srimanob · 2023-06-13T19:16:06Z

Configuration/PyReleaseValidation/python/relval_2026.py

-numWFIB.extend([21034.99,21034.999]) #2026D88 premixing combined stage1+stage2 (ttbar+PU200, ttbar+PU50 for PR test)
-numWFIB.extend([20834.21,21034.21,21034.9921]) #2026D88 prodlike, prodlike PU, prodlike premix stage1+stage2
-numWFIB.extend([21034.114]) #2026D88 PU, with 10% OT ineffiency
+numWFIB.extend([20834.501,20834.502]) #2026D88 Patatrack local reconstruction on CPU, Patatrack local reconstruction on GPU (to remove when available in D98)


Do we have .501,.502 versions in D98 yet, @AdrianoDee ?
Thx.

srimanob · 2023-06-13T19:19:07Z

test parameters:

workflows = 24834.0,24834.911,24834.103,25061.97,24834.5,24834.9,25034.99,25034.999,24834.21,25034.21,25034.9921,25034.114,24896.0,24900.0
relvals_opt = --what cleanedupgrade,standard,highstats,pileup,generator,extendedgen,production,identity,ged,machine,premix,nano,gpu,2017,2026

srimanob · 2023-06-13T19:20:17Z

@cmsbuild please test

Test all new workflows that will run on IB.

cmsbuild · 2023-06-13T20:47:08Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e4322/33132/summary.html
COMMIT: bff79cf
CMSSW: CMSSW_13_2_X_2023-06-13-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41946/33132/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals

ValueError: Undefined workflows: 24834.999

RelVals-INPUT

24896.0DAS Error
24900.0DAS Error

srimanob · 2023-06-13T20:58:35Z

The relvals of 24896.0, and 24900.0, are missing from last submissions with CMSSW_13_2_0_pre1__UPSG_Std_2026D98noPU,

Could you please take care for these two workflows, @cms-sw/pdmv-l2 ? Thanks.

cmsbuild · 2023-06-13T21:03:36Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41946/35920

This PR adds an extra 64KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/Geometry/README.md modified in PR(s): Phase 2 tracker: realistic description of 3D modules in inner tracker #41880
- File Configuration/PyReleaseValidation/python/relval_2026.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255, Phase 2 tracker: realistic description of 3D modules in inner tracker #41880
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): More Configurable Pixel Tracks and HIon Template #41632, Enable feature to recycle GEN for FastSim #41945, Add prototype HcalNanoAOD #41607
- File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): Guard for No Digis for Phase2 and Patatrack NuGun #41255, Updated nano matrix 13.2 #41867

cmsbuild · 2023-06-21T14:43:50Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e4322/33291/summary.html
COMMIT: 7dfc495
CMSSW: CMSSW_13_2_X_2023-06-20-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41946/33291/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

25.0

----- Begin Fatal Exception 21-Jun-2023 13:36:14 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

5.1

----- Begin Fatal Exception 21-Jun-2023 13:36:32 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

135.4

----- Begin Fatal Exception 21-Jun-2023 13:36:34 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log
136.79136.79_RunMuOnia2017B/step2_RunMuOnia2017B.log
136.791136.791_RunNoBPTX2017B/step2_RunNoBPTX2017B.log

Expand to see more relval errors ...

AddOn Tests

fastsim

----- Begin Fatal Exception 21-Jun-2023 13:34:18 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

fastsim1

----- Begin Fatal Exception 21-Jun-2023 13:34:18 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

fastsim2

----- Begin Fatal Exception 21-Jun-2023 13:34:18 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'Pythia8ConcurrentGeneratorFilter' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

srimanob · 2023-06-21T18:25:29Z

@cmsbuild please test

cmsbuild · 2023-06-21T21:50:26Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5e4322/33308/summary.html
COMMIT: 7dfc495
CMSSW: CMSSW_13_2_X_2023-06-21-1600/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41946/33308/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 164 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3312943
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3312921
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 231 log files, 174 edm output root files, 50 DQM output files
TriggerResults: no differences found

srimanob · 2023-06-22T04:39:50Z

+Upgrade

All D98 workflows run fine. Migration of .50X workflows in relval_2026.py will come in the follow up PR.

srimanob · 2023-06-23T08:33:47Z

Could you please consider to sign, @cms-sw/pdmv-l2 @cms-sw/geometry-l2 ?
So we can keep testing D98 for every PR. Thanks.

sunilUIET · 2023-06-23T08:45:26Z

+pdmv

civanch · 2023-06-23T10:39:54Z

+1

cmsbuild · 2023-06-23T10:40:18Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2023-06-23T11:54:21Z

+1

srimanob · 2023-06-23T14:30:25Z

Note to @cms-sw/reconstruction-l2 for updating the comparison matrix. Thx.

migrate to D98

bff79cf

srimanob marked this pull request as ready for review June 13, 2023 19:07

cmsbuild added this to the CMSSW_13_2_X milestone Jun 13, 2023

cmsbuild added geometry-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending labels Jun 13, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 13, 2023

srimanob commented Jun 13, 2023

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Jun 13, 2023

cmsbuild added tests-rejected and removed tests-started labels Jun 13, 2023

fix PU workflow

7dfc495

cmsbuild added tests-pending code-checks-pending and removed tests-rejected code-checks-approved labels Jun 13, 2023

cmsbuild added tests-rejected and removed tests-started labels Jun 21, 2023

cmsbuild added tests-started and removed tests-rejected labels Jun 21, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 21, 2023

cmsbuild added upgrade-approved and removed upgrade-pending labels Jun 22, 2023

cmsbuild mentioned this pull request Jun 22, 2023

[Bug-fix] Add NANOAOD output to RECONANORUN3 step #42047

Merged

cmsbuild added pdmv-approved and removed pdmv-pending labels Jun 23, 2023

cmsbuild added geometry-approved fully-signed and removed geometry-pending pending-signatures labels Jun 23, 2023

cmsbuild added orp-approved and removed orp-pending labels Jun 23, 2023

cmsbuild merged commit f897a20 into cms-sw:master Jun 23, 2023

srimanob deleted the 132_MigrateToPhase2D98 branch June 23, 2023 14:50

cmsbuild mentioned this pull request Jun 23, 2023

addition of 2023C WFs and modifications 2023B WFs to run with era Run3 #42069

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate Phase-2 geometry to D98 #41946

Migrate Phase-2 geometry to D98 #41946

srimanob commented Jun 13, 2023

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

srimanob Jun 13, 2023

srimanob commented Jun 13, 2023

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 21, 2023

srimanob commented Jun 21, 2023

cmsbuild commented Jun 21, 2023

srimanob commented Jun 22, 2023 •

edited

Loading

srimanob commented Jun 23, 2023

sunilUIET commented Jun 23, 2023

civanch commented Jun 23, 2023

cmsbuild commented Jun 23, 2023

rappoccio commented Jun 23, 2023

srimanob commented Jun 23, 2023

Migrate Phase-2 geometry to D98 #41946

Migrate Phase-2 geometry to D98 #41946

Conversation

srimanob commented Jun 13, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

srimanob Jun 13, 2023

Choose a reason for hiding this comment

srimanob commented Jun 13, 2023

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

RelVals

RelVals-INPUT

srimanob commented Jun 13, 2023

cmsbuild commented Jun 13, 2023

cmsbuild commented Jun 21, 2023

RelVals

RelVals-INPUT

AddOn Tests

srimanob commented Jun 21, 2023

cmsbuild commented Jun 21, 2023

Comparison Summary

srimanob commented Jun 22, 2023 • edited Loading

srimanob commented Jun 23, 2023

sunilUIET commented Jun 23, 2023

civanch commented Jun 23, 2023

cmsbuild commented Jun 23, 2023

rappoccio commented Jun 23, 2023

srimanob commented Jun 23, 2023

srimanob commented Jun 22, 2023 •

edited

Loading