Add a Phase2 HLT tracking DQM monitoring

[mmusich]

PR description:

Minimal set of changes needed to the Tracking @ HLT DQM monitoring sequence for the Phase-2 setup.
This relies on the current naming of the tracking collections for the phase-2 HLT:

• generalTracks (somewhat copied from offline)
• hltPhase2PixelTracks (pixel tracks)
• hltPhase2PixelVertices (pixel vertices)

The latter two were introduced in #42491 but the name change was not percolated to the FEVTDEBUGHLTEventContent, so it becomes necessary to include them in the list of collections to be persisted. This is done in 8b2b22d.
Changes for making the Phase-2 HLT Validation are more extensive and will be addressed with another PR at a later time.

PR validation:

Run successfully runTheMatrix.py -l 24834.0 -t 4 -j 8 --ibeos and checked that HLT tracking plots are filled.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42783/36892

• This PR adds an extra 28KB to repository

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

• Configuration/EventContent (operations)
• DQMOffline/Configuration (dqm)
• DQMOffline/Trigger (dqm)

@cmsbuild, @rappoccio, @nothingface0, @emanueleusai, @davidlange6, @pmandrik, @syuvivida, @antoniovilela, @tjavaid, @micsucmed, @fabiocos, @rvenditti can you please review it and eventually sign? Thanks.
@mtosi, @Fedespring, @Martin-Grunewald, @missirol, @HuguesBrun, @jhgoh, @threus, @trocino, @fabiocos, @cericeci, @rociovilar this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0a0d2/34743/summary.html
COMMIT: 3d5c18f
CMSSW: CMSSW_13_3_X_2023-09-13-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42783/34743/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 53 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 38 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3348561
• DQMHistoTests: Total failures: 1499
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3347040
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 200178.51600000003 KiB( 49 files compared)
• DQMHistoSizes: changed ( 23234.0,... ): 17811.328 KiB HLT/Pixel
• DQMHistoSizes: changed ( 23234.0,... ): 12250.442 KiB HLT/Tracking
• DQMHistoSizes: changed ( 23234.0,... ): 1722.700 KiB HLT/EXO
• DQMHistoSizes: changed ( 23234.0,... ): 833.110 KiB HLT/EGM
• DQMHistoSizes: changed ( 23234.0,... ): 571.370 KiB HLT/Vertexing
• DQMHistoSizes: changed ( 23234.0,... ): 134.070 KiB DQM/TimerService
• DQMHistoSizes: changed ( 23234.0,... ): 39.145 KiB HLT/LumiMonitoring
• DQMHistoSizes: changed ( 23234.0,... ): 0.570 KiB HLT/EventInfo
• DQMHistoSizes: changed ( 23234.0,... ): 0.350 KiB HLT/BTV
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[mmusich]

In the last push I add fa291dd which substitutes the Phase1 pixel cluster monitoring (useless for phase-2) and introduces the Phase-2 IT and OT cluster monitoring. Also I get rid of several other sequences that produce (currently) empty plots in the Phase-2 HLT. Respective POGs are welcome to add on to the sequence once they are ready to do so.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42783/36897

• This PR adds an extra 28KB to repository

[srimanob]

phase2_common is not more appropriate here, to represent what it will do (i.e. for future when other modules will be added back) ?
However, it should give the same result as phase2_tracker is always in phase-2 workflow.

[mmusich]

Well at the moment, it's fully tracker specific, so I'd rather not change it. In case it gets updated you can make sure the appropriate modifier is used

[srimanob]

OK, thx.

[emanueleusai]

Just for the sake of due diligence: the PR adds about 400MB of histogram memory. Is it justified?
From the code It looks like the monitoring sequence for P2 tracker is being added, so I do expect a bunch of new plots, but the PR description just mentions "minimal set of changes".

[mmusich]

Is it justified?

As mentioned in #39362 this PR addresses (partially) the lack of monitoring of HLT objects. I would say it is justified as currently there is no proper way to check that PRs acting on the phase2 HLT menu are in line with expectations. Recent history shows that there were mistakes introduced (e.g. see #39323) that would be avoidable. So this PR is desireable from the HLT point of view.

but the PR description just mentions "minimal set of changes".

Minimal in the sense that I didn't add all the possible HLT monitoring plots (that are by the way already included in Run3 relvals) for other objects, as I am not an expert in those.

[emanueleusai]

Thank you very much for the feedback

[emanueleusai]

+1

[mmusich]

@cms-sw/orp-l2 your operations signature is needed here.

[antoniovilela]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

[slava77]

@mmusich
I didn't quite follow the details, still: was the comparison wrt TP skipped because there were some issues?

[mmusich]

@slava77

was the comparison wrt TP skipped because there were some issues?

no particular issues to speak of. The changes necessary are simply more involved to ensure the right association with Phase-2 tracker hits than just adding the DQM. I feel like TRK POG at HLT would be better suited to make these changes, if at the moment there is no manpower available I can also have a look.

[slava77]

@slava77

was the comparison wrt TP skipped because there were some issues?

no particular issues to speak of. The changes necessary are simply more involved to ensure the right association with Phase-2 tracker hits than just adding the DQM. I feel like TRK POG at HLT would be better suited to make these changes, if at the moment there is no manpower available I can also have a look.

@vjmastra @lguzzi
are you available to follow up on this part (validation wrt TP)?
Please clarify.
Thank you

[mmusich]

@vjmastra @lguzzi
are you available to follow up on this part (validation wrt TP)?
Please clarify.
Thank you

as it seems this item wasn't picked up by the TRK POG @ HLT, I took the liberty of going ahead at #43094