PPS DQM plots for new RPOTs 45-220-NR-HR and 56-220-NR-HR (backport of 41389)

[grzanka]

PR description:

This PR adds new plots to the DQM for two Roman Pots (previously not included in DQM).
Is it needed in the Online DQM GUI

PR validation:

Standard runTheMatrix and manual tests on new data by @smorovic

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #41389

[cmsbuild]

A new Pull Request was created by @grzanka (Leszek Grzanka) for CMSSW_13_0_X.

It involves the following packages:

• DQM/CTPPS (dqm)
• DataFormats/OnlineMetaData (daq, reconstruction)

@micsucmed, @emanueleusai, @emeschi, @clacaputo, @cmsbuild, @syuvivida, @pmandrik, @mandrenguyen, @smorovic, @rvenditti can you please review it and eventually sign? Thanks.
@missirol, @fabferro, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from PPS DQM plots for new RPOTs 45-220-NR-HR and 56-220-NR-HR. #41389

[perrotta]

@grzanka since the master moved today to CMSSW_13_2_X, please don't forget to prepare also a backport to 13_1_X if you want this backport in 13_0_X get merged

[grzanka]

@grzanka since the master moved today to CMSSW_13_2_X, please don't forget to prepare also a backport to 13_1_X if you want this backport in 13_0_X get merged

Thanks for notice, I will prepare it soon.

[smorovic]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e9af2/32363/summary.html
COMMIT: c5d4c72
CMSSW: CMSSW_13_0_X_2023-05-03-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41518/32363/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 13 lines from the logs
• Reco comparison results: 12 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3554764
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3554732
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 797.1469999999998 KiB( 48 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): 46.891 KiB CTPPS/common
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[mandrenguyen]

backport of #41389

[mandrenguyen]

type ctpps

[mandrenguyen]

+reconstruction

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e9af2/32487/summary.html
COMMIT: 4abe20a
CMSSW: CMSSW_13_0_X_2023-05-08-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41518/32487/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3554764
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3554738
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 797.1469999999998 KiB( 48 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): 46.891 KiB CTPPS/common
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[smorovic]

+daq

Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).

Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

[grzanka]

+daq

Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).

Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

I confirm that we need that patch in the online DQM. As for the schedule, please propose something that is well timed and leads to shortest deployment time.

[mandrenguyen]

+reconstruction
resign

[smorovic]

+daq
Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).
Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

I confirm that we need that patch in the online DQM. As for the schedule, please propose something that is well timed and leads to shortest deployment time.

Today is the ORP meeting, we can bring it up. If a release is tagged on Friday (or later), I think that could work.
Early next week (Monday, possibly Tuesday) we can then bring up at the daily run meeting and see when everyone (DAQ, DQM, HLT) can move to the new version. HLT also unpacks CTPPS product (even if it doesn't use it), so it should stay in sync.
Maybe it's good to add a note also to the google doc of the weekly run meeting of today.

[grzanka]

+daq
Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).
Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

I confirm that we need that patch in the online DQM. As for the schedule, please propose something that is well timed and leads to shortest deployment time.

Today is the ORP meeting, we can bring it up. If a release is tagged on Friday (or later), I think that could work.
Early next week (Monday, possibly Tuesday) we can then bring up at the daily run meeting and see when everyone (DAQ, DQM, HLT) can move to the new version. HLT also unpacks CTPPS product (even if it doesn't use it), so it should stay in sync.
Maybe it's good to add a note also to the google doc of the weekly run meeting of today.

Will you be connected to the ORP meeting today ? I won't be able to connect unfortunately...

[smorovic]

+daq
Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).
Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

I confirm that we need that patch in the online DQM. As for the schedule, please propose something that is well timed and leads to shortest deployment time.

Today is the ORP meeting, we can bring it up. If a release is tagged on Friday (or later), I think that could work.
Early next week (Monday, possibly Tuesday) we can then bring up at the daily run meeting and see when everyone (DAQ, DQM, HLT) can move to the new version. HLT also unpacks CTPPS product (even if it doesn't use it), so it should stay in sync.
Maybe it's good to add a note also to the google doc of the weekly run meeting of today.

Will you be connected to the ORP meeting today ? I won't be able to connect unfortunately...

Yes, it's now now and we just discussed i. Release should be ready for ~ early next week (DQM will also test this patch in the meantime).

[emanueleusai]

+daq

Note, @emanueleusai, @grzanka, that we should deploy this (in DQM) only with the DAQ event-builder update that is coming next week: May 15th or later is the tentative date because our expert is at the conference (CHEP).

Therefore, merging this patch should be well timed, i.e. we should probably hold in case a new 13_0_X release or patch release is planned between today and Friday.

@smorovic thank you for the clarification, so does it make sense to test this online without the DAQ event-builder update?

[smorovic]

@emanueleusai
I would expect it to have mismatches in plotting since input is different, but should at least run without crash or exception (as the bitmask..). Maybe just running in playback and checking that the job completes cleanly?

[syuvivida]

Hi @smorovic
our software expert Michel Succar @micsucmed is testing this PR at playback and post the results here later.

Eiko

[micsucmed]

Hi @smorovic, playback runs fine with the PR, no crashes or exceptions, it seems to run cleanly.

[smorovic]

Thanks @micsucmed, then if from DQM side there is no other issue, please sign off all three PRs (13_2, 13_1 and 13_0).

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

+1