Update beamhlt client unitTests

[francescobrivio]

PR description:

This PR updates the DQM beamhlt client unitTest to use the streamer files of a 2023 pp collisions run (370580),
which have the correct event content (see cms-data/DQM-Integration#3 for details)

• The new files are being added in cms-data in Add DQMOnlineBeamspot streamer files for run 370580 cms-data/DQM-Integration#6

Additionally, in light of the issue reported in #43108, the old unitTest configuration (i.e. using the streamer files from an older run) is kept active under the name: TestDQMOnlineClient-beamhlt_dqm_sourceclient-legacy

PR validation:

Successfully ran the beamhlt unittest on the new run.

Backport:

Not a backport, no backport needed

[francescobrivio]

test parameters:

• pull_request = Add DQMOnlineBeamspot streamer files for run 370580 cms-data/DQM-Integration#6

[francescobrivio]

@cmsbuild please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43110/37359

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

• DQM/Integration (dqm)

@syuvivida, @antoniovagnerini, @rvenditti, @tjavaid, @nothingface0 can you please review it and eventually sign? Thanks.
@threus, @batinkov this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5f9ee2/35410/summary.html
COMMIT: 5633dad
CMSSW: CMSSW_13_3_X_2023-10-24-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43110/35410/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @cerminar Phase-2 L1T: Fix GT packed format for Correlator e/gamma and tau objects #43075
• @fwyzard Implement std::array-like wrapper #43076
• @kpedro88 argparse migration for cmsDriver and FWCore #43042
• @waredjeb Fix GlobalTag for dumping Fireworks Geometry #43098
• @fwyzard Mark the TriangularSolverMatrix "kernel" functions as EIGEN_DEVICE_FUNC cmsdist#8782
• @thesps Add conifer external cmsdist#8780

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5f9ee2/35410/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5f9ee2/35410/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testTauEmbeddingProducers had ERRORS

Comparison Summary

Summary:

• You potentially added 74 lines to the logs
• Reco comparison results: 7 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3351458
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3351430
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 166 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[francescobrivio]

• The failing unitTest (testTauEmbeddingProducers) is completely unrelated to this PR.
• On the contrary the beamhlt unitTests behave as expected:
  • TestDQMOnlineClient-beamhlt_dqm_sourceclient-legacy reports the same issue discussed in TestDQMOnlineClient-beamhlt_dqm_sourceclient silently fails in IBs #43108, as expected
  • TestDQMOnlineClient-beamhlt_dqm_sourceclient runs successfully on the new streamer files

[francescobrivio]

@cmsbuild please test

[cmsbuild]

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5f9ee2/35536/summary.html
COMMIT: 5633dad
CMSSW: CMSSW_13_3_X_2023-10-31-1400/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43110/35536/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

RelVals-INPUT

AddOn Tests

• read312RV

----- Begin Fatal Exception 31-Oct-2023 20:03:45 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_13_3_X_2023-10-31-1400/src/Utilities/ReleaseScripts/scripts/read312RV_cfg.py
Exception Message:
 unknown python problem occurred.
ModuleNotFoundError: No module named 'past'

At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_13_3_X_2023-10-31-1400/src/FWCore/ParameterSet/python/Types.py(6): <module>
  <frozen importlib._bootstrap>(228): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(850): exec_module
  <frozen importlib._bootstrap>(695): _load_unlocked
  <frozen importlib._bootstrap>(986): _find_and_load_unlocked
  <frozen importlib._bootstrap>(1007): _find_and_load
  /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_13_3_X_2023-10-31-1400/src/FWCore/ParameterSet/python/Config.py(15): <module>
  <frozen importlib._bootstrap>(228): _call_with_frames_removed
  <frozen importlib._bootstrap_external>(850): exec_module
  <frozen importlib._bootstrap>(695): _load_unlocked
  <frozen importlib._bootstrap>(986): _find_and_load_unlocked
  <frozen importlib._bootstrap>(1007): _find_and_load
  /cvmfs/cms-ib.cern.ch/sw/x86_64/week1/el8_amd64_gcc12/cms/cmssw/CMSSW_13_3_X_2023-10-31-1400/src/Utilities/ReleaseScripts/scripts/read312RV_cfg.py(2): <module>

----- End Fatal Exception -------------------------------------------------

• fastsim

[fastsim:1] cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi  --conditions auto:run1_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - elapsed time: 0 sec (ended on Tue Oct 31 20:03:48 2023) - exit: 256

• fastsim1

[fastsim1:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - elapsed time: 0 sec (ended on Tue Oct 31 20:03:53 2023) - exit: 256

Expand to see more addon errors ...

• fastsim2
• pat1
• hlt_mc_Fake
• hlt_mc_Fake
• hlt_mc_Fake
• hlt_mc_Fake1
• hlt_mc_Fake1
• hlt_mc_Fake1
• hlt_mc_Fake2
• hlt_mc_Fake2
• hlt_mc_Fake2
• hlt_mc_GRun
• hlt_mc_GRun
• hlt_mc_GRun
• hlt_mc_HIon
• hlt_mc_HIon
• hlt_mc_HIon
• hlt_mc_PIon
• hlt_mc_PIon
• hlt_mc_PIon
• hlt_mc_PRef
• hlt_mc_PRef
• hlt_mc_PRef
• hlt_data_Fake
• hlt_data_Fake
• hlt_data_Fake
• hlt_data_Fake1
• hlt_data_Fake1
• hlt_data_Fake1
• hlt_data_Fake2
• hlt_data_Fake2
• hlt_data_Fake2
• hlt_data_GRun
• hlt_data_GRun
• hlt_data_GRun
• hlt_data_HIon
• hlt_data_HIon
• hlt_data_HIon
• hlt_data_PIon
• hlt_data_PIon
• hlt_data_PIon
• hlt_data_PRef
• hlt_data_PRef
• hlt_data_PRef

[francescobrivio]

Again the errors are unrelated to this PR and the new unitTests keep working as expected.
@cms-sw/dqm-l2 a king ping to sign this PR and its sister PR in cms-data cms-data/DQM-Integration#6

[tjavaid]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[antoniovilela]

+1

[antoniovilela]

merge