MTD geometry: update tests to scenario D88, update DDFilteredView::parameters() accordingly

[fabiocos]

PR description:

This PR addresses the issue #36837 (comment) reported at the simulation meeting https://indico.cern.ch/event/1127757/contributions/4733494/attachments/2389568/4085879/20220211-Phase2SWPlan.pdf slide 12 by @srimanob .

All the MTD standalone geometry tests are moved to the latest D88 scenario, and in order to cope with boolean solids, the DetectorDescription/DDCMS package is update in the DDFilteredView::parameters() method, to accomodate also polycones as possible components of a boolean solid.

The unit tests need the appropriate reference cms-data/Geometry-TestReference#10 to correctly run.

PR validation:

MTD geometry unit tests run smoothly and provide the expected geometry dump (as in scenario D77 with only a z shift in ETL sensors position).

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36970/28336

• This PR adds an extra 24KB to repository

[cmsbuild]

A new Pull Request was created by @fabiocos (Fabio Cossutti) for master.

It involves the following packages:

• DetectorDescription/DDCMS (geometry)
• Geometry/MTDCommonData (geometry, upgrade)
• Geometry/MTDGeometryBuilder (geometry, upgrade)
• Geometry/MTDNumberingBuilder (geometry, upgrade)
• RecoMTD/DetLayers (upgrade, reconstruction)

@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @clacaputo, @slava77, @jpata can you please review it and eventually sign? Thanks.
@bsunanda, @slomeo, @vargasa this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[fabiocos]

please test with cms-data/Geometry-TestReference#10

[srimanob]

Hi @fabiocos
Thanks for this. Could you please advise how I can make a local test with external?

[fabiocos]

@srimanob the external is needed just to get the final comparison in the MTD unit tests correct, I run the configuration and use its output as the new reference. For any other purpose the external is useless. After this PR step2 still crashes anyway:

----- Begin Fatal Exception 15-Feb-2022 11:51:46 CET-----------------------
An exception of category 'InvalidReference' occurred while
   [0] Processing global begin Run run: 1
   [1] Prefetching for module L1FPGATrackProducer/'TTTracksFromExtendedTrackletEmulation'
   [2] Calling method for EventSetup module trackerDTC::ProducerES/'TrackTriggerSetup'
Exception Message:
NullPointer 
----- End Fatal Exception -------------------------------------------------
Another exception was caught while trying to clean up files after the primary fatal exception.

[srimanob]

Hi @fabiocos
The issue of MTD found in step-3 when I run (i) GEN-SIM with DD4hep ==> (ii) DIGI-HLT with DDD.
So I would like to test with step-3 for the next issue.

[srimanob]

Page 11-12 of https://indico.cern.ch/event/1127757/contributions/4733494/attachments/2389568/4085879/20220211-Phase2SWPlan.pdf includes the testing of running DD4hep GEN-SIM ⇒ DDD DIGI-HLT ⇒ DD4hep RECO. I will try to turn DT-alignment off and the this fixed MTD to see what is the next :)

[fabiocos]

@srimanob merging this PR should be enough, you need the external just when you issue a scram b runtests command. In principle for that it should be enough to point to the correct external copy in the cmsswdata.xml in the config section of the working area, although I did not find the correct replacement syntax for time being.

[fabiocos]

@srimanob in any case the simplest way to test also the external is to redefine CMSSW_SEARCH_PATH by prepending the directory where the new reference data files sit

[srimanob]

I've tested this PR in RECO step, we can pass issue of MTD reco geometry construction now. I've updated the doc https://docs.google.com/document/d/1es0C2gH8KVt87iDoPRpdq8VVDjuEKtdd4kPjTsfW_RU/edit?usp=sharing

[srimanob]

+Upgrade

This PR can't be tested with DD4hep workflow because we have several issues to run. To test, I've done the local test using cmsDriver.py step3 -s RAW2DIGI,L1Reco,RECO,RECOSIM,PAT --conditions auto:phase2_realistic_T21 --datatier GEN-SIM-RECO,MINIAODSIM -n 10 --eventcontent FEVTDEBUGHLT,MINIAODSIM --geometry DD4hepExtended2026D88 --era Phase2C17I13M9 --procModifiers dd4hep --python RecoGlobal_DD4hep_2026D88.py --no_exec --filein file:step2.root --fileout file:step3.root
With this PR, the job can run through reco geometry construction.

The following doc is updated,
https://docs.google.com/document/d/1es0C2gH8KVt87iDoPRpdq8VVDjuEKtdd4kPjTsfW_RU/edit?usp=sharing

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4c0807/22430/summary.html
COMMIT: 0a18925
CMSSW: CMSSW_12_3_X_2022-02-14-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36970/22430/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @pmandrik Move ecalcalib_dqm_sourceclient to Run-3 era #36958
• @bsunanda Phase2-hgx300 Fix the rotation angle of layers in V16 geometry of HGCal #36945
• @mmusich remove more CMSDEPRECATED_X warnings in the AlCa/DB area #36962
• @lecriste Pattern Recognition validation for HGCal Tracksters #36712
• @fabferro PPS fix to comply to EDModule rules #36960
• @bsunanda Run3-alca218B Clean some of the Hcal calibration code in the test area of Calibration/HcalCalibalgos #36964
• @andrea21z Adding muon MVA ID #36179
• @kakwok Set receiveMuShower =true for uGT #36929
• @dinyar Add muon shower comparison plots #36928
• @thomreis ECAL - Remove unused parameters from the multifit algorithm #36935
• @cms-sw Update tag for RecoMuon-MuonIdentification to V01-13-00 cmsdist#7626
• @cms-sw [Coral] Add const to char* to fix compiler errors with PyString_AsString cmsdist#7622

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4c0807/22430/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4c0807/22430/git-merge-result

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 8388 differences found in the comparisons
• DQMHistoTests: Total files compared: 46
• DQMHistoTests: Total histograms compared: 3757427
• DQMHistoTests: Total failures: 104335
• DQMHistoTests: Total nulls: 20456
• DQMHistoTests: Total successes: 3632614
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -56760.091 KiB( 45 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): 1.487 KiB L1TEMU/L1TdeStage2uGMT
• DQMHistoSizes: changed ( 23234.0,... ): -7096.313 KiB HGCAL/HGCalValidator
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 193 log files, 42 edm output root files, 46 DQM output files
• TriggerResults: no differences found

[cvuosalo]

How should we interpret the test results? Why were additional merge-commits included?

[perrotta]

please test
(After the troubles with previous IB, which couldn't build, since CMSSW_12_3_X_2022-02-15-2300 the situation should have returned to normal: let retest, with hopefully no other PRs added on top)

[fabiocos]

@perrotta without the external update included any MTD unit test will fail, but this does not mean it did not work, just that the final output comparison included in it was done with the incorrect (D76) reference

[perrotta]

please test with cms-data/Geometry-TestReference#10

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4c0807/22462/summary.html
COMMIT: 0a18925
CMSSW: CMSSW_12_3_X_2022-02-16-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36970/22462/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-4c0807/138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-4c0807/138.5_ExpressCollisions+RunMinimumBias2021+TIER0EXPRUN3+ALCARECOEXPR3+HARVESTDEXPR3
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-4c0807/139.001_RunMinimumBias2021+RunMinimumBias2021+HLTDR3_2021+RECODR3_MinBiasOffline+HARVESTD2021MB

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3965143
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3965113
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.004 KiB( 48 files compared)
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[cvuosalo]

+1

[clacaputo]

+reconstruction

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1