Improve caching of MF geometry across runs

[namapane]

PR description:

Added a protection to check that the MF geometry cache introduced in #38640 is still valid.
Addresses item 1) in issue #38669.

Also added a test to siumlate a job spanning over several runs with different magnet currents, triggering the cache in different conditions. It can be run with:

MagneticField/Engine/test/regression.py producerType=fromDB_DD4hep current=-1

As expected, the cache works when switching among maps with the same geometry (3, 3.5 and 3.8 T; or 0 and 2 T).
When switching between 3 and 2 T, the geometry needs to be redone, which as expected leads to a crash in dd4hep.
This can only be fixed upstream; however, this is a very exotic use case (ie job spanning different runs taken while the magnet was ramping). In any case, the test can be used to keep track of this potential issue.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38737/31051

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @namapane (Nicola Amapane) for master.

It involves the following packages:

• MagneticField/Engine (reconstruction)
• MagneticField/GeomBuilder (reconstruction)

@jpata, @cmsbuild, @clacaputo can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[clacaputo]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-dd6324/26311/summary.html
COMMIT: 9340531
CMSSW: CMSSW_12_5_X_2022-07-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38737/26311/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3662417
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3662393
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[clacaputo]

This can only be fixed upstream; however, this is a very exotic use case

Hi @namapane , just for my education, how exotic are these cases? Do we have any occurrences in the past?
The upstream fix that you are suggesting is something related to DD4HEP or to MF?

[namapane]

This can only be fixed upstream; however, this is a very exotic use case

Hi @namapane , just for my education, how exotic are these cases? Do we have any occurrences in the past? The upstream fix that you are suggesting is something related to DD4HEP or to MF?

The scenario would be: an online or offline process running over several runs taken at different values of the magnet current, which cross the 2T-3T working points. It's hard to think of a case where this may happen; it could maybe be HLT online jobs for data taken during a magnet ramp -unless jobs are started from scratch when a new run is start in this case.
Another case would be the offline processing of data taken during a magnet ramp, but given that such data would be basically useless for any practical purpose I doubt it has ever or will ever happened.

In any case, the fix would have to be in DD4HEP, to allow rebuilding a geometry in the same process. @civanch may be able to comment better.

[clacaputo]

In any case, the fix would have to be in DD4HEP, to allow rebuilding a geometry in the same process. @civanch may be able to comment better.

Hi @civanch , do you think it would be better to open an issue about it?

[civanch]

@namapane , @clacaputo , I think that it is not needed to duplicate issue #38669.
This PR is valid and should be merged. Concerning change of geometry between runs in one job: it is not recommended for Geant4 but in some cases may work by chance, for example, in the case of single thread.

[clacaputo]

+reconstruction

• address item 1 of Make a more robust fix to deal with possible multiple creations of the MF geometries #38669
• no reco chages

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1