[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

sunilUIET · 2023-06-26T09:56:32Z

Add 2023 WFs and fix the reco step to store Nano output.
This is a partial backport of #42069 and #42047

There is a change in the HLT step as well to have the correct era/conditions for 2022 WFs starting from 139.001

cmsbuild · 2023-06-26T09:56:59Z

A new Pull Request was created by @sunilUIET (sunil bansal) for CMSSW_13_0_X.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@cmsbuild, @bbilin, @srimanob, @sunilUIET, @AdrianoDee can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @Martin-Grunewald, @missirol, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from addition of 2023C WFs and modifications 2023B WFs to run with era Run3 #42069

sunilUIET · 2023-06-26T09:57:11Z

est parameters:

relvals_opt= -w standard
workflows = 141.001,141.002,141.003,141.004,141.005,141.006,141.007,141.008,141.009,141.010,141.011,141.012,141.013,141.031,141.032,141.033,141.034,141.035,141.036,141.037,141.038

sunilUIET · 2023-06-26T09:57:29Z

@cmsbuild please test

cmsbuild · 2023-06-26T12:19:38Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa5c1d/33387/summary.html
COMMIT: 744e2cb
CMSSW: CMSSW_13_0_X_2023-06-26-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42089/33387/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 15 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3317136
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3317111
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-06-26T19:13:52Z

Pull request #42089 was updated. @cmsbuild, @bbilin, @srimanob, @sunilUIET, @AdrianoDee can you please check and sign again.

cmsbuild · 2023-06-26T19:23:27Z

Pull request #42089 was updated. @cmsbuild, @bbilin, @srimanob, @sunilUIET, @AdrianoDee can you please check and sign again.

sunilUIET · 2023-06-26T19:24:04Z

test parameters:

relvals_opt= -w standard
workflows = 141.001,141.002,141.003,141.004,141.005,141.006,141.007,141.008,141.009,141.010,141.011,141.012,141.013,141.031,141.032,141.033,141.034,141.035,141.036,141.037,141.038

sunilUIET · 2023-06-26T19:24:25Z

@cmsbuild please test

cmsbuild · 2023-06-26T23:57:52Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa5c1d/33398/summary.html
COMMIT: 781fa71
CMSSW: CMSSW_13_0_X_2023-06-26-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42089/33398/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@sroychow [13.0.X] Fixed channel decoding for the timeout error in SiPixel RawToDigi #42033
@jordan-martins Secondary Dataset backport 130X #42084
@mmusich [13.0.X] add specialized ZeroBias relval steps and use them for 2022 data workflows, in order to employ @rerecoZeroBias DQM sequence #42063
@makortel [13_0_X] Protect assignments of StripDetset() in StMeasurementDetSet::getDetSet() #42042

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa5c1d/33398/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-aa5c1d/33398/git-merge-result

Comparison Summary

Summary:

You potentially removed 259 lines from the logs
Reco comparison results: 1428 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3317136
DQMHistoTests: Total failures: 2558
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3314556
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: found differences in 1 / 47 workflows

sunilUIET · 2023-06-27T07:53:50Z

+pdmv

sunilUIET · 2023-06-28T07:52:44Z

@srimanob do see the issue now.

perrotta · 2023-06-29T08:31:08Z

@sunilUIET when you are ready with this backport please don't forget to prepare also the PR to backport in 13_1_X

srimanob · 2023-06-29T12:32:48Z

@srimanob do see the issue now.

Is this PR ready to go, or you need to fix?

sunilUIET · 2023-06-29T12:35:31Z

@srimanob I fixed it. I was wondering if you still see the issue you mentioned earlier. Sorry I could not check it from the logs

srimanob · 2023-06-29T12:43:09Z

I think if you summarize on 139.001 then we should be good to go. @sunilUIET

sunilUIET · 2023-06-29T12:51:21Z

+pdmv

srimanob · 2023-07-03T17:16:46Z

Sorry @sunilUIET
Do you get a clear answer on 139.001? If yes, could you please update the PR description, then this PR is good to go. Thanks.

sunilUIET · 2023-07-04T03:25:30Z

Sorry @sunilUIET Do you get a clear answer on 139.001? If yes, could you please update the PR description, then this PR is good to go. Thanks.

@srimanob it is done

perrotta · 2023-07-04T05:23:26Z

backport of #42069
(partial backport; it also includes previous #41629 and the bug fixes in #42047)

srimanob · 2023-07-04T05:41:36Z

+Upgrade

cmsbuild · 2023-07-04T05:42:02Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-07-04T05:59:23Z

@sunilUIET this still misses the backport to 13_1_X: let wait for it being at least submitted before merging this PR in the older 13_0_X cycle

perrotta · 2023-07-04T17:01:35Z

+1

partial backport of addition of 2023C WFs and modifications 2023B WFs to run with era Run3 #42069
partial backport of Add 2023 data relval workflows #41629
backport of the bug fixes in [Bug-fix] Add NANOAOD output to RECONANORUN3 step #42047)

sunilUIET · 2023-07-05T02:20:33Z

Hi @perrotta I am not fully available this week. I will work on the backport next week.

perrotta · 2023-07-05T05:57:09Z

@sunilUIET all newly added worklows are failing in the CMSSW_13_0_X_2023-07-04-2300 IB
I am going to revert this PR in 13_0_X so that we can build 13_0_10, see #42187

srimanob · 2023-07-05T12:12:03Z

There was a typo MINIAOOD, which should be MINIAOD.
I clone this PR and fix the typo in #42191

srimanob · 2023-07-05T12:13:13Z

Configuration/PyReleaseValidation/python/relval_steps.py

-steps['RECONANORUN3_reHLT']=merge([{'-s':'RAW2DIGI,L1Reco,RECO,PAT,NANO,DQM:@standardDQM+@miniAODDQM+@nanoAODDQM'},steps['RECODR3_reHLT']])
-steps['RECONANORUN3_ZB_reHLT']=merge([{'-s':'RAW2DIGI,L1Reco,RECO,PAT,NANO,DQM:@rerecoZeroBias+@miniAODDQM+@nanoAODDQM'},steps['RECODR3_reHLT']])
-steps['RECOCOSMRUN3_reHLT']=merge([{'--scenario':'cosmics','-s':'RAW2DIGI,L1Reco,RECO,DQM','--datatier':'RECO,DQMIO','--eventcontent':'RECO,DQM'},steps['RECONANORUN3_reHLT']])
+steps['RECONANORUN3_reHLT']=merge([{'-s':'RAW2DIGI,L1Reco,RECO,PAT,NANO,DQM:@standardDQMFakeHLT+@miniAODDQM+@nanoAODDQM','--datatier':'RECO,MINIAOD,NANOAOD,DQMIO','--eventcontent':'RECO,MINIAOOD,NANOEDMAOD,DQM'},steps['RECODR3_reHLT']])


Typo here MINIAOOD

add 2023 WFs, fix NANO step

744e2cb

cmsbuild added this to the CMSSW_13_0_X milestone Jun 26, 2023

cmsbuild added pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending labels Jun 26, 2023

cmsbuild added tests-started and removed tests-pending labels Jun 26, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 26, 2023

minor fixes

7c9f537

cmsbuild added tests-pending and removed tests-approved labels Jun 26, 2023

cmsbuild added tests-started and removed tests-pending labels Jun 26, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 26, 2023

cmsbuild added pdmv-approved and removed pdmv-pending labels Jun 27, 2023

cmsbuild added the tests-approved label Jun 27, 2023

cmsbuild added pdmv-approved and removed pdmv-pending labels Jun 29, 2023

cmsbuild added the backport-ok label Jul 4, 2023

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Jul 4, 2023

cmsbuild added orp-approved and removed orp-pending labels Jul 4, 2023

cmsbuild merged commit d94217b into cms-sw:CMSSW_13_0_X Jul 4, 2023

perrotta mentioned this pull request Jul 5, 2023

Revert "[13_0_X] Addition of 2023 WFs and Fixing NANO step" #42187

Merged

srimanob mentioned this pull request Jul 5, 2023

[13_0_X] Addition of 2023 WFs and Fixing NANO step (Fix typo) #42191

Merged

srimanob reviewed Jul 5, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

sunilUIET commented Jun 26, 2023 •

edited

Loading

cmsbuild commented Jun 26, 2023 •

edited

Loading

sunilUIET commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

sunilUIET commented Jun 27, 2023

sunilUIET commented Jun 28, 2023

perrotta commented Jun 29, 2023

srimanob commented Jun 29, 2023

sunilUIET commented Jun 29, 2023

srimanob commented Jun 29, 2023

sunilUIET commented Jun 29, 2023

srimanob commented Jul 3, 2023

sunilUIET commented Jul 4, 2023

perrotta commented Jul 4, 2023 •

edited

Loading

srimanob commented Jul 4, 2023

cmsbuild commented Jul 4, 2023

perrotta commented Jul 4, 2023 •

edited

Loading

perrotta commented Jul 4, 2023

sunilUIET commented Jul 5, 2023

perrotta commented Jul 5, 2023

srimanob commented Jul 5, 2023

srimanob Jul 5, 2023

[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

[13_0_X] Addition of 2023 WFs and Fixing NANO step #42089

Conversation

sunilUIET commented Jun 26, 2023 • edited Loading

cmsbuild commented Jun 26, 2023 • edited Loading

sunilUIET commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

Comparison Summary

cmsbuild commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

sunilUIET commented Jun 26, 2023

cmsbuild commented Jun 26, 2023

Comparison Summary

sunilUIET commented Jun 27, 2023

sunilUIET commented Jun 28, 2023

perrotta commented Jun 29, 2023

srimanob commented Jun 29, 2023

sunilUIET commented Jun 29, 2023

srimanob commented Jun 29, 2023

sunilUIET commented Jun 29, 2023

srimanob commented Jul 3, 2023

sunilUIET commented Jul 4, 2023

perrotta commented Jul 4, 2023 • edited Loading

srimanob commented Jul 4, 2023

cmsbuild commented Jul 4, 2023

perrotta commented Jul 4, 2023 • edited Loading

perrotta commented Jul 4, 2023

sunilUIET commented Jul 5, 2023

perrotta commented Jul 5, 2023

srimanob commented Jul 5, 2023

srimanob Jul 5, 2023

Choose a reason for hiding this comment

sunilUIET commented Jun 26, 2023 •

edited

Loading

cmsbuild commented Jun 26, 2023 •

edited

Loading

perrotta commented Jul 4, 2023 •

edited

Loading

perrotta commented Jul 4, 2023 •

edited

Loading