Added more ZDC functionality to DQM #46478

cfmcginn · 2024-10-22T13:48:26Z

PR description:

This PR is to replace existing, closed PR here: #45948

The PR adds to the DQM for ZDC specific histograms, via ZDCQIE10Task

This PR is updated relative to the closed previous PR, mostly fixing build issue, retitling some histograms, and updating python such that emulated ZDC sums fill. Another PR exists where it is rebased to 14_1_X for backport, currently open here #46407.

PR validation:

The PR is tested in CMSSW_14_2_X_2024-10-20-0000, after running a merge-topic to incorporate the PR into a fresh area
scram b ran successfully, validating the build

Following test command produces the expected output:

cmsRun DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py inputFiles=/eos/cms/store/group/dpg_hcal/comm_hcal/PFG/backup_raw/HIHLTPhysics-HIRun2023A-RAW-v1-375754.root runkey=hi_run

under the upload directory, if using the offline candidate GT. Inspected histograms inside the DQM file appear reasonable for ZDC sums.

In addition, the following set of commands are run
scram b runtests
scram build code-checks
scram build code-format

code-checks revealed no issues, code-format offered no suggestions. runtests had multiple failures, detailed below - however, every failure existed already with runtests in a fresh area, and matches the set of failures documented already in the backport. As such, it is unclear the failures are the result of the PR

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

As mentioned above, this PR is to replace the closed PR #45948, and is paired to the backport to CMSSW_14_1_X, found here #46407

cfmcginn · 2024-10-22T13:48:38Z

@hjbossi @mandrenguyen

cmsbuild · 2024-10-22T13:48:53Z

cms-bot internal usage

cmsbuild · 2024-10-22T13:49:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46478/42334

cmsbuild · 2024-10-22T13:50:22Z

A new Pull Request was created by @cfmcginn for master.

It involves the following packages:

DQM/HcalTasks (dqm)
DQM/Integration (dqm)

@antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please review it and eventually sign? Thanks.
@DryRun, @abdoulline, @batinkov, @bsunanda, @denizsun, @francescobrivio, @salimcerci, @threus this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

antoniovagnerini · 2024-10-22T13:59:35Z

please test

DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py

cmsbuild · 2024-10-22T16:20:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46478/42338

cmsbuild · 2024-10-22T16:21:05Z

Pull request #46478 was updated. @antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.

antoniovagnerini · 2024-10-23T08:47:16Z

please test

cmsbuild · 2024-10-23T12:52:40Z

-1

Failed Tests: RelVals-INPUT
Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8bbce2/42351/summary.html
COMMIT: ab400b6
CMSSW: CMSSW_14_2_X_2024-10-22-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46478/42351/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

2024.0000012024.000001_RunJetMET02024D_10k/step1_dasquery.log
2024.000001DAS Error

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3566331
DQMHistoTests: Total failures: 416
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3565895
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: no differences found

mmusich · 2024-10-23T08:48:40Z

DQM/HcalTasks/interface/ZDCQIE10Task.h

@@ -9,24 +9,83 @@
 */

 #include "DQM/HcalCommon/interface/ElectronicsMap.h"
+#include "DQM/HcalTasks/interface/DigiTask.h"


could this be alpha-ordered?

It appears this is also an unnecessary header, and is removed

mmusich · 2024-10-23T08:48:55Z

DQM/HcalTasks/interface/ZDCQIE10Task.h

+#include "DQM/HcalCommon/interface/Constants.h"
+#include "FWCore/Framework/interface/Event.h"
+#include "FWCore/Framework/interface/MakerMacros.h"
+#include "FWCore/Framework/interface/one/EDAnalyzer.h"


Appears not needed - removed in latest commit

mmusich · 2024-10-23T08:49:03Z

DQM/HcalTasks/interface/ZDCQIE10Task.h


-class ZDCQIE10Task : public DQMEDAnalyzer {
+#include "FWCore/ServiceRegistry/interface/Service.h"
+#include "CommonTools/UtilAlgos/interface/TFileService.h"


Unneeded - removed in latest commit

mmusich · 2024-10-23T14:17:17Z

DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py

 process.hcalDigis.InputLabel = rawTag
 process.emulTPDigisNoTDCCut = process.emulTPDigis.clone(
     parameters = cms.untracked.PSet(
 	ADCThresholdHF = cms.uint32(255),
 	TDCMaskHF = cms.uint64(0xFFFFFFFFFFFFFFFF)
     )
 )
-process.HcalTPGCoderULUT.LUTGenerationMode = False
+process.HcalTPGCoderULUT.LUTGenerationMode = True


is this ultimately really needed given the answer at https://github.com/cms-sw/cmssw/pull/46407/files#r1804677477 ?

This should be reverted to false, ZDC channels were now added to HcalL1TriggerObjects, fixing the need to set this to true.

I can make the reversion to False requested, but I would like to also understand the failed check first and fix everything in one go - if I click through on the error details I can't actually navigate the page after to find anything. Any guidance on where to look to fix this failed check is appreciated

I would like to also understand the failed check first and fix everything in one go

nothing failed, these were transient das errors.

Ah! Apologies, I did not understand that - I will revert to False

Reverted to false in latest commit

cmsbuild · 2024-10-24T18:36:31Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46478/42361

cmsbuild · 2024-10-24T18:36:53Z

Pull request #46478 was updated. @antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.

mmusich · 2024-10-24T18:37:31Z

please test

cmsbuild · 2024-10-24T21:48:08Z

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8bbce2/42384/summary.html
COMMIT: d85a6cb
CMSSW: CMSSW_14_2_X_2024-10-24-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46478/42384/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3566343
DQMHistoTests: Total failures: 508
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3565815
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: no differences found

cfmcginn · 2024-10-25T08:49:48Z

If useful, just want to note that to run the offline test command

 cmsRun DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py inputFiles=/eos/cms/store/group/dpg_hcal/comm_hcal/PFG/backup_raw/HIHLTPhysics-HIRun2023A-RAW-v1-375754.root runkey=hi_run

I have to make some changes to the python (things we have reverted like specifying the candidate global tag), but with the latest commits removing all the headers it still runs and produces what appears to be reasonable output

antoniovagnerini · 2024-10-25T10:21:38Z

+1

spurious differences from HGCal DQM test (open issue Non reproducibility in HGCal DQM #46416)

cmsbuild · 2024-10-25T10:22:01Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-10-25T10:48:36Z

+1

Added more ZDC functionality to DQM

6200a84

cmsbuild added this to the CMSSW_14_2_X milestone Oct 22, 2024

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending code-checks-pending labels Oct 22, 2024

cfmcginn mentioned this pull request Oct 22, 2024

[14_1_X] Added more ZDC functionality to DQM #46407

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 22, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 22, 2024

mmusich reviewed Oct 22, 2024

View reviewed changes

DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py Outdated Show resolved Hide resolved

Update per PR feedback, largely comment removal + GT fixes

ab400b6

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Oct 22, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 22, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 23, 2024

cmsbuild added tests-rejected and removed tests-started labels Oct 23, 2024

mmusich reviewed Oct 23, 2024

View reviewed changes

remove unneeded headers + revert py bool to false

d85a6cb

cmsbuild added tests-pending code-checks-pending and removed tests-rejected code-checks-approved labels Oct 24, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 24, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 24, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 24, 2024

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Oct 25, 2024

cmsbuild added orp-approved and removed orp-pending labels Oct 25, 2024

cmsbuild merged commit 9daf432 into cms-sw:master Oct 25, 2024
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added more ZDC functionality to DQM #46478

Added more ZDC functionality to DQM #46478

cfmcginn commented Oct 22, 2024

cfmcginn commented Oct 22, 2024

cmsbuild commented Oct 22, 2024 •

edited

Loading

cmsbuild commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

antoniovagnerini commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

antoniovagnerini commented Oct 23, 2024

cmsbuild commented Oct 23, 2024

mmusich Oct 23, 2024

cfmcginn Oct 24, 2024

mmusich Oct 23, 2024

cfmcginn Oct 24, 2024

mmusich Oct 23, 2024

cfmcginn Oct 24, 2024

mmusich Oct 23, 2024

hjbossi Oct 23, 2024

cfmcginn Oct 24, 2024

mmusich Oct 24, 2024

cfmcginn Oct 24, 2024

cfmcginn Oct 24, 2024

cmsbuild commented Oct 24, 2024

cmsbuild commented Oct 24, 2024

mmusich commented Oct 24, 2024

cmsbuild commented Oct 24, 2024

cfmcginn commented Oct 25, 2024

antoniovagnerini commented Oct 25, 2024

cmsbuild commented Oct 25, 2024

mandrenguyen commented Oct 25, 2024

Added more ZDC functionality to DQM #46478

Added more ZDC functionality to DQM #46478

Conversation

cfmcginn commented Oct 22, 2024

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cfmcginn commented Oct 22, 2024

cmsbuild commented Oct 22, 2024 • edited Loading

cmsbuild commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

antoniovagnerini commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

cmsbuild commented Oct 22, 2024

antoniovagnerini commented Oct 23, 2024

cmsbuild commented Oct 23, 2024

RelVals-INPUT

Comparison Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Oct 24, 2024

cmsbuild commented Oct 24, 2024

mmusich commented Oct 24, 2024

cmsbuild commented Oct 24, 2024

Comparison Summary

cfmcginn commented Oct 25, 2024

antoniovagnerini commented Oct 25, 2024

cmsbuild commented Oct 25, 2024

mandrenguyen commented Oct 25, 2024

cmsbuild commented Oct 22, 2024 •

edited

Loading