Update TSG tests and Relval Steps to use new run3_[hlt|data]_relval GTs

[Martin-Grunewald]

PR description:

Update TSG tests and Relval Steps to use new run3_[hlt|data]_relval GTs

PR validation:

TSG tests

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

[Martin-Grunewald]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37184/28759

• This PR adds an extra 52KB to repository

[cmsbuild]

A new Pull Request was created by @Martin-Grunewald (Martin Grunewald) for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)
• HLTrigger/Configuration (hlt)

@jordan-martins, @AdrianoDee, @bbilin, @wajidalikhan, @Martin-Grunewald, @missirol, @srimanob, @kskovpen can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @missirol, @silviodonato, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[missirol]

One minor comment: as far as I understand, 138.3 does not need run3_data_relval.

> runTheMatrix.py -nel 138.3 

138.3  RunMinimumBias2021Splash+COPYPASTER3+RECODR3Splash+HARVESTDR3 [1]: input from: /MinimumBias/Commissioning2021-v1/RAW with run [] 
                                           [2]: cmsDriver.py step2  -s NONE --conditions auto:run1_mc --output '[{"t":"RAW","e":"ALL"}]' --customise_commands "process.ALLRAWoutput.fastCloning=cms.untracked.bool(False)" --era Run3 -n 100 
                                           [3]: cmsDriver.py step3  --conditions auto:run3_data_relval -s RAW2DIGI,L1Reco,RECO,PAT,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+EcalESAlign,DQM:@standardDQMFakeHLT+@miniAODDQM --datatier RECO,MINIAOD,DQMIO --eventcontent RECO,MINIAOD,DQM --data  --process reRECO --scenario pp --era Run3 --customise Configuration/DataProcessing/RecoTLR.customisePostEra_Run3 -n 2
                                           [4]: cmsDriver.py step4  -s HARVESTING:@standardDQMFakeHLT+@miniAODDQM --conditions auto:run3_data_relval --data  --filetype DQM --scenario pp --era Run3 -n 100 

It could be fixed with this patch [1], but I think this is something that could be fixed later on without delaying this PR.

[1]

diff --git a/Configuration/PyReleaseValidation/python/relval_steps.py b/Configuration/PyReleaseValidation/python/relval_steps.py
index c6e9035e761..c90402fad6b 100644
--- a/Configuration/PyReleaseValidation/python/relval_steps.py
+++ b/Configuration/PyReleaseValidation/python/relval_steps.py
@@ -2246,6 +2246,7 @@ steps['RECODR3']=merge([{'--scenario':'pp',
                          '--customise':'Configuration/DataProcessing/RecoTLR.customisePostEra_Run3'},dataReco])
 
 steps['RECODR3Splash']=merge([{'-n': 2,
+                               '--conditions':'auto:run3_data',
                                '-s': 'RAW2DIGI,L1Reco,RECO,PAT,ALCA:SiStripCalZeroBias+SiStripCalMinBias+TkAlMinBias+EcalESAlign,DQM:@standardDQMFakeHLT+@miniAODDQM'
                               },steps['RECODR3']])

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c273e/22974/summary.html
COMMIT: 85dac94
CMSSW: CMSSW_12_3_X_2022-03-08-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37184/22974/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3703700
• DQMHistoTests: Total failures: 11
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3703666
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0 KiB( 0 files compared)
• Checked 0 log files, 0 edm output root files, 49 DQM output files

[Martin-Grunewald]

+1

[srimanob]

@cmsbuild please test

Just re-trigger the test, to get proper bin-by-bin DQM comparison.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8c273e/22995/summary.html
COMMIT: 85dac94
CMSSW: CMSSW_12_3_X_2022-03-09-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37184/22995/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3695161
• DQMHistoTests: Total failures: 10
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3695129
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[srimanob]

@Martin-Grunewald
What will be the relation between run3_[hlt|data] and run3_[hlt|data]_relval? Will ones with _relval be the last developed menu?
Thx.

[Martin-Grunewald]

run3_[hlt|data] contain in the GT the L1T menu used when actually taking the data.
run3_[hlt|data]_relval contain in the GT a more recent L1T menu, used to run relvals with more recent HLT menu in CMSSW in turn using the seeds of the new L1T menu, on older data.

We had the same in Run-2: run2_[hlt|data] and run2_[hlt|data]_relval for Run-2

[srimanob]

+Upgrade

[missirol]

@cms-sw/pdmv-l2 , could you please review this short PR?

[kskovpen]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1