Add tracking efficiencies w.r.t. offline to HLT monitoring client #38959

kjpena · 2022-08-03T10:27:53Z

PR description:

This PR adds the sequence producing HLT tracking efficiencies and fake rates w.r.t. offline tracks to the HLT monitoring client

PR validation:

Efficiency and fake rate plots are now produced when running the harvesting step with @HLTMon

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

A backport to 12_4_X will be needed

cmsbuild · 2022-08-03T10:34:00Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38959/31405

This PR adds an extra 12KB to repository

cmsbuild · 2022-08-03T10:34:18Z

A new Pull Request was created by @kjpena for master.

It involves the following packages:

DQMOffline/Configuration (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@threus, @rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

missirol · 2022-08-03T11:29:24Z

DQMOffline/Configuration/python/DQMOffline_SecondStep_cff.py

@@ -98,7 +98,7 @@
                                         DQMMessageLoggerClientSeq )


-HLTMonitoringClient = cms.Sequence(trackingMonitorClientHLT * trackingForDisplacedJetMonitorClientHLT)
+HLTMonitoringClient = cms.Sequence(trackingMonitorClientHLT * trackEfficiencyMonitoringClientHLT * trackingForDisplacedJetMonitorClientHLT)
 HLTMonitoringClientPA= cms.Sequence(trackingMonitorClientHLT * PAtrackingMonitorClientHLT)


Would it make sense to update HLTMonitoringClientPA in the same way?

It could be useful but, since that's the client used in the Heavy-Ions scenario, I would let @FHead comment

@FHead , there is a question on a HLT-DQM sequence for HIon. Do you know who could provide feedback to @kjpena ?

(cross-posted..)

Let me tag relevant HI tracking and HLT people @abaty @CesarBernardes @denerslemos to see if they have inputs

@abaty @CesarBernardes @denerslemos

Do you have any inputs?

Sorry for the delay. Since pA is similar to pp, I think would be good to include trackEfficiencyMonitoringClientHLT in HLTMonitoringClientPA. Thanks

Thanks for the input @denerslemos ~~I will go ahead and update the HLTMonitoringClientPA~~

@denerslemos, please note that HI sequences will not be updated in this PR, as discussed below. If needed, we can follow up with a separate PR

@kjpena, ok, thank you so much for the efforts. I think it's a great idea to follow up using a separate PR.

mmusich · 2022-08-03T13:45:10Z

please test

cmsbuild · 2022-08-03T17:48:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d9eb9/26625/summary.html
COMMIT: 48c09da
CMSSW: CMSSW_12_5_X_2022-08-03-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38959/26625/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3691510
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3691480
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

mmusich · 2022-08-05T08:20:24Z

Thanks @kjpena, would you mind preparing a backport for 12.4.x for this PR (that would be ~ urgent)?
I think the discussion about HI sequences that unfortunately landed here #38959 (comment)
won't become relevant until several months.
Thanks!

cmsbuild · 2022-08-05T14:46:32Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38959/31429

This PR adds an extra 16KB to repository

cmsbuild · 2022-08-05T14:46:53Z

Pull request #38959 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

missirol · 2022-08-05T14:58:45Z

DQMOffline/Configuration/python/DQMOffline_SecondStep_cff.py

-HLTMonitoringClient = cms.Sequence(trackingMonitorClientHLT * trackingForDisplacedJetMonitorClientHLT)
-HLTMonitoringClientPA= cms.Sequence(trackingMonitorClientHLT * PAtrackingMonitorClientHLT)
+HLTMonitoringClient = cms.Sequence(trackingMonitorClientHLT * trackEfficiencyMonitoringClientHLT * trackingForDisplacedJetMonitorClientHLT)
+HLTMonitoringClientPA= cms.Sequence(trackingMonitorClientHLT * trackEfficiencyMonitoringClientHLT * PAtrackingMonitorClientHLT)


I think one would also have to add hltToOfflineTrackValidatorSequence to OfflineHLTMonitoringPA in order to produce the histograms used by trackEfficiencyMonitoringClientHLT in HLTMonitoringClientPA.

In any case, since this PR is now considered urgent (#38959 (comment)), feel free to postpone HIon-related updates.

Yes. In fact, one would need a dedicated sequence, similar to hltToOfflineTrackValidatorSequence but with the collections that need to be monitored in the Heavy-Ions scenario. I would indeed suggest to leave these changes for a separate PR, either by or with further input from Heavy-Ions experts

emanueleusai · 2022-08-06T04:30:13Z

please test

cmsbuild · 2022-08-06T08:31:25Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d9eb9/26671/summary.html
COMMIT: a7fed5e
CMSSW: CMSSW_12_5_X_2022-08-05-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38959/26671/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3691612
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3691588
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

mmusich · 2022-08-08T21:32:42Z

urgent

its backport Adding tracking efficiencies w.r.t. offline to HLT monitoring client [124X] #38973 is needed for data-taking

cmsbuild · 2022-08-08T22:42:07Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d9eb9/26711/summary.html
COMMIT: 48c09da
CMSSW: CMSSW_12_5_X_2022-08-08-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38959/26711/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3691612
DQMHistoTests: Total failures: 14
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3691576
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

emanueleusai · 2022-08-10T04:55:39Z

+1

cmsbuild · 2022-08-10T04:56:02Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-08-10T06:13:30Z

+1

Adding tracking efficiencies w.r.t. offline to HLT monitoring client

48c09da

cmsbuild added this to the CMSSW_12_5_X milestone Aug 3, 2022

cmsbuild added code-checks-pending dqm-pending orp-pending pending-signatures tests-pending labels Aug 3, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 3, 2022

missirol reviewed Aug 3, 2022

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Aug 3, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 3, 2022

kjpena mentioned this pull request Aug 5, 2022

Adding tracking efficiencies w.r.t. offline to HLT monitoring client [124X] #38973

Merged

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels Aug 5, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 5, 2022

missirol reviewed Aug 5, 2022

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Aug 6, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 6, 2022

kjpena force-pushed the EfficienciesWRTOffline branch from a7fed5e to 48c09da Compare August 8, 2022 18:59

cmsbuild added tests-started and removed tests-approved labels Aug 8, 2022

cmsbuild added the urgent label Aug 8, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 8, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Aug 10, 2022

cmsbuild added orp-approved and removed orp-pending labels Aug 10, 2022

cmsbuild merged commit 4c7b44b into cms-sw:master Aug 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add tracking efficiencies w.r.t. offline to HLT monitoring client #38959

Add tracking efficiencies w.r.t. offline to HLT monitoring client #38959

kjpena commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

missirol Aug 3, 2022

kjpena Aug 3, 2022

missirol Aug 3, 2022 •

edited

Loading

FHead Aug 3, 2022

missirol Aug 5, 2022

denerslemos Aug 5, 2022

kjpena Aug 5, 2022 •

edited

Loading

kjpena Aug 8, 2022

denerslemos Aug 8, 2022

mmusich commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

mmusich commented Aug 5, 2022

cmsbuild commented Aug 5, 2022

cmsbuild commented Aug 5, 2022

missirol Aug 5, 2022

kjpena Aug 8, 2022

emanueleusai commented Aug 6, 2022

cmsbuild commented Aug 6, 2022

mmusich commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

emanueleusai commented Aug 10, 2022

cmsbuild commented Aug 10, 2022

qliphy commented Aug 10, 2022

Add tracking efficiencies w.r.t. offline to HLT monitoring client #38959

Add tracking efficiencies w.r.t. offline to HLT monitoring client #38959

Conversation

kjpena commented Aug 3, 2022

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

missirol Aug 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjpena Aug 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmusich commented Aug 3, 2022

cmsbuild commented Aug 3, 2022

Comparison Summary

mmusich commented Aug 5, 2022

cmsbuild commented Aug 5, 2022

cmsbuild commented Aug 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emanueleusai commented Aug 6, 2022

cmsbuild commented Aug 6, 2022

Comparison Summary

mmusich commented Aug 8, 2022

cmsbuild commented Aug 8, 2022

Comparison Summary

emanueleusai commented Aug 10, 2022

cmsbuild commented Aug 10, 2022

qliphy commented Aug 10, 2022

missirol Aug 3, 2022 •

edited

Loading

kjpena Aug 5, 2022 •

edited

Loading