Add NANO step and Mini+Nano validations to FastSim workflows #38834

srimanob · 2022-07-22T18:56:18Z

PR description:

This PR updates the FastSim steps in runTheMatrix following the presentation at the SIM meeting on July 15 by

Adding NANO step
- By doing this, we need to split GEN step out from the main FastSim sequence. This PR then allows to reuse GEN datasets in future for FastSim validation (few lines of code needed to be enable when datasets are available).
Adding MiniAOD validation (as in Add miniAODDQM FastSim Validation #38736, closed)
- Few modifications on POG validation sequences are needed to support FastSim without HLT
Adding NanoAOD validation
Updating harvesting step to get less complain on missing histograms in FastSim

Note that this PR can take over #38736

Examples of DQM from this PR: https://tinyurl.com/2yb7zl22

MiniAOD and NanoAOD folders appears as expected. However, POG experts should check if something is missing.
Compare to https://tinyurl.com/2ykx26v2 which come without this PR.

Note:

There is a warning from JetMET analyzer as it looks for HLT. I assume this is a task of JetMET to modify the code to accept NoHLT from FastSim.

%MSG-e HLTConfigProvider:   JetAnalyzer:jetDQMAnalyzerAk4PFCHSPuppiCleanedMiniAOD@streamBeginRun  23-Jul-2022 14:31:36 CEST Run: 1 Stream: 0
Falling back to ProcessName-only init using ProcessName 'HLT' !

This PR removes the following sequences from Tau validation sequence when fastsim era is used, to support FastSim/MC. This should be reviewed by Tau expert.

tauValidationMiniAODRealData,tauValidationMiniAODRealElectronsData,tauValidationMiniAODRealMuonsData
produceDenominatorRealData,produceDenominatorRealElectronsData,produceDenominatorRealMuonsData

The follow issue should be followed up, to review how to use validation and dqm sequences in MC relvals, Review of VALIDATION and DQM sequences in RelVals MC #38828

PR validation:

Run the following configs our from runTheMatrix give expected results:
runTheMatrix.py --what upgrade -l 13234.0,13434.0,13240.303 -t 8 --wm init
DQM harvesting output is uploaded to GUI, to check.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

No need of backport.

cmsbuild · 2022-07-22T19:03:04Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38834/31216

This PR adds an extra 68KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760, Add wfs to test Run3 skims #38830, Don't recluster jets in HI miniAOD for Run 3 #38711, Add miniAODDQM FastSim Validation #38736
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760, Clean up Phase-2 geometry (Up to D85) #38702, Add run3 tracking low pu era #33532
- File DQMOffline/Configuration/python/DQMOfflineFS_cff.py modified in PR(s): Add miniAODDQM FastSim Validation #38736
- File DQMOffline/Muon/python/muonAnalyzer_cff.py modified in PR(s): Add miniAODDQM FastSim Validation #38736

cmsbuild · 2022-07-22T19:03:27Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)
DQMOffline/Configuration (dqm)
DQMOffline/Muon (dqm)

@jordan-martins, @bbilin, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @kskovpen, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@sscruz, @barvic, @trtomei, @calderona, @makortel, @threus, @slomeo, @abbiendi, @Fedespring, @fabiocos, @Martin-Grunewald, @bellan, @cericeci, @HuguesBrun, @jhgoh, @ptcox, @missirol, @trocino, @beaucero, @kpedro88, @rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

srimanob · 2022-07-22T19:04:54Z

@cmsbuild please test

cmsbuild · 2022-07-22T22:57:55Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2224c4/26401/summary.html
COMMIT: 2cd92df
CMSSW: CMSSW_12_5_X_2022-07-22-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38834/26401/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

13234.0DAS Error
13240.303DAS Error
13434.0DAS Error

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3618350
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3618320
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 206 log files, 45 edm output root files, 51 DQM output files
TriggerResults: no differences found

kskovpen · 2022-07-29T08:36:46Z

+pdmv

srimanob · 2022-07-29T08:53:19Z

@cmsbuild please test

Just to refresh the test.

cmsbuild · 2022-07-29T12:46:28Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2224c4/26524/summary.html
COMMIT: 26b433b
CMSSW: CMSSW_12_5_X_2022-07-28-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38834/26524/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3579916
DQMHistoTests: Total failures: 14
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3579880
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 206 log files, 45 edm output root files, 51 DQM output files
TriggerResults: found differences in 3 / 48 workflows

civanch · 2022-07-29T12:47:00Z

+1

perrotta · 2022-07-30T12:13:17Z

This PR removes the following sequences from Tau validation sequence, to supprot FastSim/MC. This should be reviewed by Tau expert.
tauValidationMiniAODRealData,tauValidationMiniAODRealElectronsData,tauValidationMiniAODRealMuonsData
produceDenominatorRealData,produceDenominatorRealElectronsData,produceDenominatorRealMuonsData

Could @cms-sw/tau-pog-l2 either approve or comment about the removal of the Tau validation sequences in FastSim?

perrotta · 2022-07-30T12:07:34Z

Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

@@ -148,6 +148,8 @@ def condition(self, fragment, stepList, key, hasHarvest):
    def preventReuse(self, stepName, stepDict, k):
        if "Sim" in stepName:
            stepDict[stepName][k] = None
+        if "Gen" in stepName:


elif would have been more appropriate here, but it doesn't really matter here

Wrong comment before, deleted. Yes, this can change to elif.

srimanob · 2022-07-30T12:29:06Z

This PR removes the following sequences from Tau validation sequence, to supprot FastSim/MC. This should be reviewed by Tau expert.
tauValidationMiniAODRealData,tauValidationMiniAODRealElectronsData,tauValidationMiniAODRealMuonsData
produceDenominatorRealData,produceDenominatorRealElectronsData,produceDenominatorRealMuonsData
Could @cms-sw/tau-pog-l2 either approve or comment about the removal of the Tau validation sequences in FastSim?

I realize that I did not make it clear from the original PR description. The removal is done only when fastsim era is used. So, there will be no effect on FullSim or Data validations. Good to hear from @cms-sw/tau-pog-l2

danielwinterbottom · 2022-08-03T08:26:38Z

Hi, from the TauPOG side we are happy with removing these 6 sequences for FastSim - the sequences in question are intended only for data validation and therefore not needed for FastSim

qliphy · 2022-08-03T08:48:10Z

+1

cmsbuild · 2022-08-03T08:48:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

srimanob · 2022-08-03T08:53:45Z

Hi, from the TauPOG side we are happy with removing these 6 sequences for FastSim - the sequences in question are intended only for data validation and therefore not needed for FastSim

Thanks for the comment @danielwinterbottom
In fact, the sequences we are talking about seems to run also on MC FullSim RelVals. I have opened an issue to follow up at #38828.

Norraphat added 3 commits July 22, 2022 18:50

rebase: add miniaod dqm to FastSim

3592f3f

rebase

6a38d1e

Add nano to FastSim

2cd92df

srimanob marked this pull request as ready for review July 22, 2022 18:56

cmsbuild added this to the CMSSW_12_5_X milestone Jul 22, 2022

cmsbuild added code-checks-pending dqm-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jul 22, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2022

srimanob changed the title ~~[Draft] Add Nano and NanoValidation to FastSim~~ [Draft] Add NANO step and Mini+Nano validations in FastSim Jul 22, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 22, 2022

This was referenced Jul 22, 2022

Add wfs to test Run3 skims #38830

Merged

Add miniAODDQM FastSim Validation #38736

Closed

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2022

Fix INPUT, add miniaodValidation

26b433b

cmsbuild added code-checks-pending operations-pending and removed tests-rejected code-checks-approved labels Jul 23, 2022

cmsbuild added the upgrade-approved label Jul 29, 2022

cmsbuild added pdmv-approved and removed pdmv-pending labels Jul 29, 2022

cmsbuild added tests-started and removed tests-approved labels Jul 29, 2022

cmsbuild mentioned this pull request Jul 29, 2022

Add run3 PPS TDC and SAMPIC relvals #38893

Merged

cmsbuild added tests-approved and removed tests-started labels Jul 29, 2022

cmsbuild added fastsim-approved and removed fastsim-pending labels Jul 29, 2022

perrotta reviewed Jul 30, 2022

View reviewed changes

cmsbuild mentioned this pull request Jul 31, 2022

Migration of input collection parameters to untracked for CTPPS and DTMonitorModule #38854

Closed

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending pending-signatures orp-pending labels Aug 3, 2022

cmsbuild merged commit 20d46ab into cms-sw:master Aug 3, 2022

srimanob mentioned this pull request Dec 7, 2022

[Backport] Run-3 FastSim Mini and Nano workflow for validation #40256

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NANO step and Mini+Nano validations to FastSim workflows #38834

Add NANO step and Mini+Nano validations to FastSim workflows #38834

srimanob commented Jul 22, 2022 •

edited

Loading

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

kskovpen commented Jul 29, 2022

srimanob commented Jul 29, 2022

cmsbuild commented Jul 29, 2022

civanch commented Jul 29, 2022

perrotta commented Jul 30, 2022

perrotta Jul 30, 2022 •

edited

Loading

srimanob Jul 30, 2022

srimanob commented Jul 30, 2022

danielwinterbottom commented Aug 3, 2022

qliphy commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

srimanob commented Aug 3, 2022

Add NANO step and Mini+Nano validations to FastSim workflows #38834

Add NANO step and Mini+Nano validations to FastSim workflows #38834

Conversation

srimanob commented Jul 22, 2022 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

RelVals-INPUT

Comparison Summary

kskovpen commented Jul 29, 2022

srimanob commented Jul 29, 2022

cmsbuild commented Jul 29, 2022

Comparison Summary

civanch commented Jul 29, 2022

perrotta commented Jul 30, 2022

perrotta Jul 30, 2022 • edited Loading

Choose a reason for hiding this comment

srimanob Jul 30, 2022

Choose a reason for hiding this comment

srimanob commented Jul 30, 2022

danielwinterbottom commented Aug 3, 2022

qliphy commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

srimanob commented Aug 3, 2022

srimanob commented Jul 22, 2022 •

edited

Loading

perrotta Jul 30, 2022 •

edited

Loading