Add new SimBeamSpotHLLHC CondFormat #43186

francescobrivio · 2023-11-03T22:31:42Z

PR description:

First step of resolving cms-AlCaDB/AlCaTools#95:

Added a new SimBeamSpotHLLHC CondFormat and relative record
- Also added all the needed utilities to write/read/plot/inspect/test the new object
Updated HLLHCEvtVtxGenerator plugin adding the possibility to read the smearing parameters from the DB
Removed the CrabKissing parameters:
- Removing HLLHC_CK and HLLHC_CK14TeV scenarios which are not used anywhere in CMSSW and are not even supported by any c++ code since A correct description of the HL-LHC beamspot #16942

PR validation:

Code compiles and I ran successfully a few upgrade workflows.

Backport:

Not a backport, no backport needed.

francescobrivio · 2023-11-03T22:35:57Z

@cmsbuild please test

cmsbuild · 2023-11-03T22:41:57Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/37510

This PR adds an extra 80KB to repository

cmsbuild · 2023-11-03T22:42:18Z

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

CondCore/BeamSpotPlugins (db)
CondCore/Utilities (db)
CondFormats/BeamSpotObjects (alca, db)
CondFormats/DataRecord (alca, db)
CondTools/BeamSpot (alca, db)
IOMC/EventVertexGenerators (simulation)

@perrotta, @civanch, @francescobrivio, @saumyaphor4252, @consuegs, @mdhildreth can you please review it and eventually sign? Thanks.
@seemasharmafnal, @grzanka, @missirol, @tocheng, @PonIlya, @rsreds, @mmusich, @fabiocos, @yuanchao this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2023-11-04T00:18:40Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-03-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testTauEmbeddingProducers had ERRORS

RelVals

136.7611

----- Begin Fatal Exception 04-Nov-2023 00:30:42 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.7611_RunJetHT2016EreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

136.8311

----- Begin Fatal Exception 04-Nov-2023 00:30:43 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.8311_RunJetHT2017FreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

136.88811

----- Begin Fatal Exception 04-Nov-2023 00:30:44 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.88811_RunJetHT2018DreMINIAODUL


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log

Expand to see more relval errors ...

francescobrivio · 2023-11-05T16:00:02Z

@cmsbuild please test

cmsbuild · 2023-11-05T18:49:20Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35613/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-05-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43186/35613/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 361 lines from the logs
Reco comparison results: 134 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363010
DQMHistoTests: Total failures: 1787
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3361201
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

civanch · 2023-11-06T08:15:08Z

+1

smuzaffar · 2023-11-06T16:51:56Z

Milestone for this pull request has been moved to CMSSW_14_0_X.Please open a backport if it should also go in to CMSSW_13_3_X.

cmsbuild · 2024-07-04T10:32:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40812

This PR adds an extra 96KB to repository

cmsbuild · 2024-07-04T10:33:05Z

Pull request #43186 was updated. @antoniovilela, @civanch, @cmsbuild, @davidlange6, @fabiocos, @mdhildreth, @rappoccio can you please check and sign again.

francescobrivio · 2024-07-04T10:35:30Z

@cmsbuild please test

cmsbuild · 2024-07-04T16:53:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/40211/summary.html
COMMIT: 17721fb
CMSSW: CMSSW_14_1_X_2024-07-03-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43186/40211/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

There are some workflows for which there are errors in the baseline:
29634.911 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially removed 16 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 10879 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3353315
DQMHistoTests: Total failures: 52886
DQMHistoTests: Total nulls: 207
DQMHistoTests: Total successes: 3300202
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 203 log files, 168 edm output root files, 48 DQM output files
TriggerResults: found differences in 3 / 46 workflows

civanch · 2024-07-04T17:01:39Z

+1

comparison differences are expected because of minor change in formulas.

antoniovilela · 2024-07-04T18:45:25Z

+1

cmsbuild · 2024-07-04T18:45:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_13_3_X milestone Nov 3, 2023

cmsbuild added simulation-pending alca-pending db-pending pending-signatures tests-pending orp-pending code-checks-pending tracking labels Nov 3, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 3, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Nov 4, 2023

francescobrivio mentioned this pull request Nov 5, 2023

Add SimBeamSpotObjects to MC Global Tags cms-AlCaDB/AlCaTools#86

Closed

8 tasks

cmsbuild added tests-started and removed tests-rejected labels Nov 5, 2023

cmsbuild added tests-approved and removed tests-started labels Nov 5, 2023

cmsbuild added simulation-approved and removed simulation-pending labels Nov 6, 2023

cmsbuild added simulation-pending tests-pending code-checks-pending and removed simulation-approved tests-approved code-checks-approved labels Jul 4, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 4, 2024

cmsbuild added tests-started and removed tests-pending labels Jul 4, 2024

cmsbuild added tests-approved and removed tests-started labels Jul 4, 2024

cmsbuild added simulation-approved and removed simulation-pending labels Jul 4, 2024

cmsbuild added operations-approved fully-signed orp-approved and removed operations-pending pending-signatures orp-pending labels Jul 4, 2024

cmsbuild merged commit 55cba4e into cms-sw:master Jul 4, 2024
11 checks passed

cmsbuild mentioned this pull request Jul 5, 2024

Updated root to 6.30.08 cms-sw/cmsdist#9277

Merged

francescobrivio deleted the alca-add_SimBeamSpotHLLHCObjects branch July 29, 2024 08:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new SimBeamSpotHLLHC CondFormat #43186

Add new SimBeamSpotHLLHC CondFormat #43186

francescobrivio commented Nov 3, 2023 •

edited

Loading

francescobrivio commented Nov 3, 2023

cmsbuild commented Nov 3, 2023

cmsbuild commented Nov 3, 2023 •

edited

Loading

cmsbuild commented Nov 4, 2023

francescobrivio commented Nov 5, 2023

cmsbuild commented Nov 5, 2023

civanch commented Nov 6, 2023

smuzaffar commented Nov 6, 2023

cmsbuild commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

francescobrivio commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

civanch commented Jul 4, 2024

antoniovilela commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

Add new SimBeamSpotHLLHC CondFormat #43186

Add new SimBeamSpotHLLHC CondFormat #43186

Conversation

francescobrivio commented Nov 3, 2023 • edited Loading

PR description:

PR validation:

Backport:

francescobrivio commented Nov 3, 2023

cmsbuild commented Nov 3, 2023

cmsbuild commented Nov 3, 2023 • edited Loading

cmsbuild commented Nov 4, 2023

Unit Tests

RelVals

RelVals-INPUT

francescobrivio commented Nov 5, 2023

cmsbuild commented Nov 5, 2023

Comparison Summary

civanch commented Nov 6, 2023

smuzaffar commented Nov 6, 2023

cmsbuild commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

francescobrivio commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

Comparison Summary

civanch commented Jul 4, 2024

antoniovilela commented Jul 4, 2024

cmsbuild commented Jul 4, 2024

francescobrivio commented Nov 3, 2023 •

edited

Loading

cmsbuild commented Nov 3, 2023 •

edited

Loading