Phase2 HLT pixel modules duplicate removal (13_1_X backport) #42512

lguzzi · 2023-08-08T14:17:51Z

this is a backport of #42491.

PR description:

This PR removes some duplicated pixel modules from the phase2 menu and renames pixel[...] to hltPhase2Pixel[...]. More details in the master PR.
No change is expected. Some automated tests may fails because the pixelTracks and pixelVertices collections have been renamed to hltPhase2PixelTracks and hltPhase2PixelVertices as discussed in #42491 (comment).

PR validation:

tested privately running

cmsDriver.py Phase2 -s HLT:75e33 --processName=HLTX \
  --conditions auto:phase2_realistic_T21 \
  --geometry Extended2026D95 \
  --era Phase2C17I13M9 \
  --customise SLHCUpgradeSimulations/Configuration/aging.customise_aging_1000 \
  --eventcontent FEVTDEBUGHLT \
  --filein=/store/mc/Phase2Spring23DIGIRECOMiniAOD/TT_TuneCP5_14TeV-powheg-pythia8/GEN-SIM-DIGI-RAW-MINIAOD/PU200_Trk1GeV_131X_mcRun4_realistic_v5-v1/30000/01607282-0427-4687-a122-ef0a41220590.root \
  -n 10 --nThreads 4

on 13_1_X. Output: results (baseline results as run on d7d5f11)

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is a backport of #42491, needed for the integration in the phase2 HLT.

…hs (pixelFitter)

…hs (pixelFilter)

…rAll

…ingRegions and rename to hltPhase2PixelTracksAndHighPtStepTrackingRegions

cmsbuild · 2023-08-08T14:18:19Z

A new Pull Request was created by @lguzzi (Luca Guzzi) for CMSSW_13_1_X.

It involves the following packages:

HLTrigger/Configuration (hlt)

@cmsbuild, @missirol, @Martin-Grunewald can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @silviodonato, @SohamBhattacharya, @beaucero this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Phase2 HLT pixel modules duplicate removal #42491

missirol · 2023-08-08T14:21:46Z

backport of #42491

missirol · 2023-08-08T14:22:02Z

please test

cmsbuild · 2023-08-08T15:33:09Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2768f7/34168/summary.html
COMMIT: cb9eedb
CMSSW: CMSSW_13_1_X_2023-08-08-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42512/34168/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

23634.0

----- Begin Fatal Exception 08-Aug-2023 17:08:02 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'HLT_IsoMu24_FromL1TkMuon'
   [2] Prefetching for module HLTMuonGenericFilter/'hltL3crIsoL1TkSingleMu22L3f24QL3pfecalIsoFiltered0p41'
   [3] Prefetching for module MuonHLTEcalPFClusterIsolationProducer/'hltPhase2L3MuonsEcalIsodR0p3dRVeto0p000'
   [4] Calling method for module FixedGridRhoProducerFastjet/'hltFixedGridRhoFastjetAllCaloForMuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Candidate
Looking for module label: hltTowerMakerForAll
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

23834.999

----- Begin Fatal Exception 08-Aug-2023 17:17:40 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'HLT_IsoMu24_FromL1TkMuon'
   [2] Prefetching for module HLTMuonGenericFilter/'hltL3crIsoL1TkSingleMu22L3f24QL3pfecalIsoFiltered0p41'
   [3] Prefetching for module MuonHLTEcalPFClusterIsolationProducer/'hltPhase2L3MuonsEcalIsodR0p3dRVeto0p000'
   [4] Calling method for module FixedGridRhoProducerFastjet/'hltFixedGridRhoFastjetAllCaloForMuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Candidate
Looking for module label: hltTowerMakerForAll
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

23634.911

----- Begin Fatal Exception 08-Aug-2023 17:20:09 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'HLT_IsoMu24_FromL1TkMuon'
   [2] Prefetching for module HLTMuonGenericFilter/'hltL3crIsoL1TkSingleMu22L3f24QL3pfecalIsoFiltered0p41'
   [3] Prefetching for module MuonHLTEcalPFClusterIsolationProducer/'hltPhase2L3MuonsEcalIsodR0p3dRVeto0p000'
   [4] Calling method for module FixedGridRhoProducerFastjet/'hltFixedGridRhoFastjetAllCaloForMuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Candidate
Looking for module label: hltTowerMakerForAll
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

24034.0DAS Error

missirol · 2023-08-08T15:55:55Z

These errors seem related to the fact that #41749 was not backported to CMSSW_13_1_X.

@SohamBhattacharya , is the plan to backport to 13_1_X all the HLT-Phase2-menu PRs of the last weeks (or only some PRs, like this one) ?

SohamBhattacharya · 2023-08-09T07:37:19Z

These errors seem related to the fact that #41749 was not backported to CMSSW_13_1_X.

@SohamBhattacharya , is the plan to backport to 13_1_X all the HLT-Phase2-menu PRs of the last weeks (or only some PRs, like this one) ?

All the duplicate removal PRs need to be backported to 13_1_X as the plan is to run the timing measurement with a 13_1_X release.
I had missed at #41749 was not in 13_1.
@wonpoint4 can you please make a backport of the PR to 13_1_X?

missirol · 2023-08-09T08:23:37Z

FYI, #42351 and #42352 were not backported either.

missirol · 2023-08-14T13:28:20Z

please test with #42555

cmsbuild · 2023-08-14T17:49:12Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2768f7/34267/summary.html
COMMIT: cb9eedb
CMSSW: CMSSW_13_1_X_2023-08-14-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42512/34267/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

You potentially added 19 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 151 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3186945
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3186916
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: found differences in 5 / 46 workflows

missirol · 2023-08-15T05:14:28Z

please test

#42555 should now be in the IB.

cmsbuild · 2023-08-15T08:50:38Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2768f7/34276/summary.html
COMMIT: cb9eedb
CMSSW: CMSSW_13_1_X_2023-08-14-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42512/34276/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 10 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 147 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3186945
DQMHistoTests: Total failures: 108
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3186815
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

missirol · 2023-08-15T09:09:13Z

+hlt

(Spurious DQM differences in some Phase-2 wfs, compatible with #39754.)

cmsbuild · 2023-08-15T09:09:36Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-08-16T08:31:49Z

+1

lguzzi added 13 commits August 8, 2023 15:26

remove fully duplicated pixel modules from modules and paths

5b7bc38

rename pixel to hltPhase2Pixel and remove duplicated modules from pat…

abe6d39

…hs (pixelFitter)

rename pixel to hltPhase2Pixel and remove duplicated modules from pat…

0d2dea6

…hs (pixelFilter)

remove caloTowerForTrk duplicated module in favour of hltTowerMakerFo…

b187583

…rAll

rename hltTowerMakerForAll to hltPhase2TowerMakerForAll

7904df8

remove highPtTripletStepTrackingRegions in favour of pixelTracksTrack…

227a7b0

…ingRegions and rename to hltPhase2PixelTracksAndHighPtStepTrackingRegions

remove leftover duplicated modules

685918e

change name pixelTracksHitDoublets -> hltPhase2PixelTracksHitDoublets

204a4e1

rename pixelTracksHitSeeds->hltPhase2PixelTracksHitSeeds

c71b549

rename pixelTracksSeedLayers->hltPhase2PixelTracksSeedLayers

8f8d956

rename pixelTracks->hltPhase2PixelTracks

bcd9870

rename pixelVertices->hltPhase2PixelVertices

e963b1e

rename pixelTracksTask->hltPhase2PixelTracksTask

cb9eedb

cmsbuild added this to the CMSSW_13_1_X milestone Aug 8, 2023

cmsbuild added hlt-pending pending-signatures tests-pending orp-pending labels Aug 8, 2023

cmsbuild added tests-started backport-ok and removed tests-pending labels Aug 8, 2023

cmsbuild added tests-rejected and removed tests-started labels Aug 8, 2023

This was referenced Aug 12, 2023

Cleanup of HLT Phase-2 menu (#42351, #42352) [13_2_X] #42554

Merged

Cleanup of HLT Phase-2 menu (#41749, #42351, #42352) [13_1_X] #42555

Merged

cmsbuild added requires-external tests-started and removed tests-rejected labels Aug 14, 2023

cmsbuild added tests-rejected and removed tests-started labels Aug 14, 2023

cmsbuild added tests-started and removed requires-external tests-rejected labels Aug 15, 2023

cmsbuild added tests-approved and removed tests-started labels Aug 15, 2023

cmsbuild added hlt-approved fully-signed and removed hlt-pending pending-signatures labels Aug 15, 2023

missirol mentioned this pull request Aug 15, 2023

Non-reproducibility in JetMET/{Jet,MET}Validation histograms in phase2 workflows #39754

Open

cmsbuild added orp-approved and removed orp-pending labels Aug 16, 2023

cmsbuild merged commit 3c82d5f into cms-sw:CMSSW_13_1_X Aug 16, 2023

mmusich mentioned this pull request Sep 19, 2023

Add a Phase2 HLT tracking DQM monitoring (13.1.X) #42827

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Phase2 HLT pixel modules duplicate removal (13_1_X backport) #42512

Phase2 HLT pixel modules duplicate removal (13_1_X backport) #42512

lguzzi commented Aug 8, 2023

cmsbuild commented Aug 8, 2023 •

edited

Loading

missirol commented Aug 8, 2023

missirol commented Aug 8, 2023

cmsbuild commented Aug 8, 2023

missirol commented Aug 8, 2023

SohamBhattacharya commented Aug 9, 2023

missirol commented Aug 9, 2023

missirol commented Aug 14, 2023

cmsbuild commented Aug 14, 2023

missirol commented Aug 15, 2023

cmsbuild commented Aug 15, 2023

missirol commented Aug 15, 2023 •

edited

Loading

cmsbuild commented Aug 15, 2023

perrotta commented Aug 16, 2023

Phase2 HLT pixel modules duplicate removal (13_1_X backport) #42512

Phase2 HLT pixel modules duplicate removal (13_1_X backport) #42512

Conversation

lguzzi commented Aug 8, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Aug 8, 2023 • edited Loading

missirol commented Aug 8, 2023

missirol commented Aug 8, 2023

cmsbuild commented Aug 8, 2023

RelVals

RelVals-INPUT

missirol commented Aug 8, 2023

SohamBhattacharya commented Aug 9, 2023

missirol commented Aug 9, 2023

missirol commented Aug 14, 2023

cmsbuild commented Aug 14, 2023

RelVals-INPUT

Comparison Summary

missirol commented Aug 15, 2023

cmsbuild commented Aug 15, 2023

Comparison Summary

missirol commented Aug 15, 2023 • edited Loading

cmsbuild commented Aug 15, 2023

perrotta commented Aug 16, 2023

cmsbuild commented Aug 8, 2023 •

edited

Loading

missirol commented Aug 15, 2023 •

edited

Loading