FastSim premixing workflows and matrix cleanup

[kpedro88]

PR description:

1. Add premixing variations for Run 3 FastSim in the upgrade matrix.
2. Remove some spurious workflows:
   a. Some special workflow modifications get applied without actually changing any steps, leading to a pointless clone of the main workflow. These can just be filtered out globally.
   b. A few special workflows were being applied to all fragments unnecessarily. I have restricted these to just ttbar.

PR validation:

1. Checked the list of workflows in the upgrade matrix and the steps that they contain.
2. Ran the new FastSim premixing workflows successfully.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This will be backported to 14_0_X for profiling in the production release.

attn: @sbein @srimanob

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46007/41808

[cmsbuild]

A new Pull Request was created by @kpedro88 for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)

@AdrianoDee, @cmsbuild, @kskovpen, @miquork, @srimanob, @subirsarkar, @sunilUIET can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @makortel, @missirol, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[kpedro88]

test parameters:
workflow = 13461.97,13434.99
relvals_opt = --what cleanedupgrade,standard,highstats,pileup,generator,extendedgen,production,identity,ged,machine,premix,nano,gpu,2017,2026

[kpedro88]

please test

[cmsbuild]

+1

Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61b447/41539/summary.html
COMMIT: 5a61648
CMSSW: CMSSW_14_2_X_2024-09-15-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46007/41539/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 94 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 1358 differences found in the comparisons
• DQMHistoTests: Total files compared: 45
• DQMHistoTests: Total histograms compared: 3383020
• DQMHistoTests: Total failures: 3237
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3379763
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
• Checked 197 log files, 166 edm output root files, 45 DQM output files
• TriggerResults: no differences found

[kpedro88]

rebased (again)

[kpedro88]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46007/42069

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, CMSSW Integration of LST #45117, Introduce support for cluster splitting and JetCore iteration in Phase-2 track reconstruction #46001, Enable DNN supercluster in Phase2 HLT #46010

[cmsbuild]

Pull request #46007 was updated. @AdrianoDee, @Moanwar, @kskovpen, @miquork, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

[cmsbuild]

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61b447/41951/summary.html
COMMIT: 1ca6072
CMSSW: CMSSW_14_2_X_2024-10-03-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46007/41951/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 109 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 1464 differences found in the comparisons
• DQMHistoTests: Total files compared: 45
• DQMHistoTests: Total histograms compared: 3383200
• DQMHistoTests: Total failures: 6350
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3376830
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
• Checked 197 log files, 166 edm output root files, 45 DQM output files
• TriggerResults: no differences found

[kpedro88]

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 please sign

[AdrianoDee]

+pdmv

[Moanwar]

+Upgrade

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @mandrenguyen, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

+1