HCAL: update of DPGAnalysis/HcalTools for 2023 Run3

[zhokin2]

Concerning RMT and CMT. These are mostly semi-privately run macros. We use remote monitoring tool(RMT) for analysis of local hcal data to see suspesion channels. In framework of RMT the phi-symmetry monitoring(PSM) is also available. PSM is developed for global runs to facilitate calibration of hcal data by method of moments and iterative method. CMT - certification monitoring tool is developed to facilitate certification of hcal data. This tool analyses in details behaviour of estimators as function of lumi-section. This tool produces as plots and tables. These information is presenting in dedicated sites for analysis by certification experts.

CMT(Alexander Zhokin): added graphs of charge and time correlation relative to GlobalRMT,
this applies to common code and macros;
tiny update to condor related scripts;
see examples here:
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalRMT/GLOBAL_362596/HE_Tile.html

RMT(Olga Kodolova): update on scripts to run crontab using local runs of
type MIX_LED and MIX_PEDESTAL;
see examples here:
https://cms-hcal-dpg.web.cern.ch/cms-hcal-dpg/HcalRemoteMonitoring/RMT/MIXED_LED_363217/MAP.html

talk of Olga Kodolova on calibration meeting in Sep. 2022:
https://indico.cern.ch/event/1195494/contributions/5025090/attachments/2504140/4302195/Monitoring.pdf

my talk on HCAL DPG meeting in Nov. 2022:
https://indico.cern.ch/event/1082970/contributions/4565247/attachments/2328302/3966806/HCAL-DPG-Workshop2021.pdf

PR validation:

scram build code-checks

scram build code-format

scram b

the CMT, RMT and PSM tests were done locally and with use of cms-sites :

https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/RMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/CMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalRMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/ALCARECOPSM
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/IMPSM -->

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40656/33985

• This PR adds an extra 268KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40656/33985/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40656/33985/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[zhokin2]

some files to be removed...

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40656/33986

• This PR adds an extra 276KB to repository

[cmsbuild]

A new Pull Request was created by @zhokin2 (zhokin) for master.

It involves the following packages:

• DPGAnalysis/HcalTools (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[emanueleusai]

type hcal

[emanueleusai]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2364a3/30349/summary.html
COMMIT: 37a8b8c
CMSSW: CMSSW_13_0_X_2023-02-01-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40656/30349/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 1 lines to the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3555495
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3555470
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 211 log files, 162 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

The code is rather convoluted, and the warnings issued by the static analyzer may point to some possible fix/improvement to DPGAnalysis/HcalTools/plugins/CMTRawAnalyzer.cc
In any case, these are mostly private scripts intended for HCAL monitoring. Provided @cms-sw/hcal-dpg-l2 confirm that they are good as such, this PR can be merged in CMSSW

[abdoulline]

@perrotta Hi Andrea, I suppose so.
I mean Alexandr (@zhokin2) certainly needs to look at the warnings
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2364a3/30349/llvm-analysis ->
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2364a3/30349/llvm-analysis/runStaticChecks.log
may be just "a posteriori" ?

[perrotta]

Thank you @abdoulline
Sure, improvements are welcome in new possible PRs. Before adding 7k lines of new code to CMSSW I just wanted to get confirmed that this was done in agreement with the HCAL DPG. As far as I understand this is the case: fixes and adjustments can get provided in a followup PR.

[perrotta]

+1