HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35806

abdoulline · 2021-10-24T11:25:25Z

PR description:

M3 signal fit is an auxiliary one (MAHI is default), but it stays activated in HCAL Offline reconstruction (while swithed off since long time for HLT) and it wasn't explcitly looked at in the context of Run3 preparations and it's not used by any downstream consumers.
HCAL is now discussing to possibly use corresponding HBHERecHit member (auxEnergy) for saving next-to-trigger BX (SOI+1 in HCAL notations) energy from MAHI for LLP studies.

NB: this PR (once backported) serves as a workaround for recently reported issue #35785
while (in parallel) HCAL DPG figures out what went wrong with M3 as reported here:
https://hypernews.cern.ch/HyperNews/CMS/get/tier0-Ops/2294.html

PR validation:

runTheMatrix.py -l limited

cmsbuild · 2021-10-24T11:31:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35806/26169

This PR adds an extra 28KB to repository

cmsbuild · 2021-10-24T11:31:49Z

A new Pull Request was created by @abdoulline (Salavat Abdullin) for master.

It involves the following packages:

RecoEgamma/EgammaHLTProducers (hlt)
RecoLocalCalo/HcalRecProducers (reconstruction)

@jpata, @missirol, @cmsbuild, @Martin-Grunewald, @slava77 can you please review it and eventually sign? Thanks.
@Sam-Harper, @HuguesBrun, @calderona, @silviodonato, @jainshilpi, @Fedespring, @lgray, @apsallid, @sscruz, @bsunanda, @afiqaize, @wrtabb, @mariadalfonso, @ram1123, @varuns23, @cericeci, @sobhatta this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

perrotta · 2021-10-24T12:06:29Z

please test

cmsbuild · 2021-10-24T15:53:39Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-986f2b/19870/summary.html
COMMIT: 795dd3c
CMSSW: CMSSW_12_1_X_2021-10-24-0000/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35806/19870/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 74 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 2751113
DQMHistoTests: Total failures: 102
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2750988
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 39 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 170 log files, 37 edm output root files, 40 DQM output files
TriggerResults: no differences found

Martin-Grunewald · 2021-10-25T08:10:21Z

RecoEgamma/EgammaHLTProducers/python/HLTEgPhaseIITestSequence_cff.py

@@ -2654,3 +2654,6 @@
    PixelCPE = cms.string('PixelCPEGeneric'),
    StripCPE = cms.string('StripCPEfromTrackAngle')
 )
+
+from Configuration.Eras.Modifier_run3_common_cff import run3_common
+run3_common.toModify(hltHbhereco, algorithm = dict(useM3 = False))


HLT does not use Eras and anway has useM3 = cms.bool( False ) so this is not needed for HLT.

@Martin-Grunewald thank you for calrification, I (naively) thought this might be a kind of standalone config used for some purposes. I'll remove this modification.

Sorry, I see that that file (PhaseII) actually sets useM3 = True which seems nonsensical but then it has to be switched to false after all. Apologies for the confusion!

@Martin-Grunewald just to avoid any (more) confusion: do you suggest to bring the (just removed) customization back ?

Yes, but also I want Andrea Bocci @fwyzard to clarify why in that Phase-II file useM3 = True is used.

It (useM3=True) should in principle be harmless for HLT, it just takes unnecessary additional CPU time (~10% of MAHI in CPU case), as useMahi =True is set anyway and it means HCAL RecHits energy comes from MAHI, and M3 "just" fills an auxiliary energy word...

The hardcoded Phase2 copy seems a copy of the offline Run2 sequence.
processQIE11 = cms.bool(True),
processQIE8 = cms.bool(True)

we will not have QIE8 in the Phase2

And these are very expensive CPU wise, in fact were mostly calculated offline in Run2 not at HLT.
setPulseShapeFlagsQIE8 = cms.bool(True),
setLegacyFlagsQIE8 = cms.bool(True)
setNegativeFlagsQIE8 = cms.bool(True),
setNoiseFlagsQIE8 = cms.bool(True),
setPulseShapeFlagsQIE8 = cms.bool(True),

@mariadalfonso thanks for spotting these obsolete settings, I must admit I didn't even look at them, as I've been focused just on M3...

I suppose you mean that even if there is no QIE8 Digi collection in Phase2, setting all those modules (and then calling them for nothing) may take some CPU time?

cmsbuild · 2021-10-25T08:27:08Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35806/26176

This PR adds an extra 32KB to repository

cmsbuild · 2021-10-25T16:09:29Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-986f2b/19904/summary.html
COMMIT: f190b38
CMSSW: CMSSW_12_1_X_2021-10-24-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35806/19904/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

2018.1

----- Begin Fatal Exception 25-Oct-2021 17:15:25 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'prevalidation_step'
   [2] Prefetching for module MultiTrackValidator/'trackValidatorBuilding'
   [3] Prefetching for module TrackProducer/'detachedTripletStepTracks'
   [4] Prefetching for module TrackCandidateProducer/'detachedTripletStepTrackCandidates'
   [5] Prefetching for module FastTrackerRecHitMaskProducer/'detachedTripletStepMasks'
   [6] Prefetching for module TrackProducer/'detachedQuadStepTracks'
   [7] Prefetching for module TrackCandidateProducer/'detachedQuadStepTrackCandidates'
   [8] Prefetching for module FastTrackerRecHitMaskProducer/'detachedQuadStepMasks'
   [9] Prefetching for module TrackProducer/'lowPtTripletStepTracks'
   [10] Prefetching for module TrackCandidateProducer/'lowPtTripletStepTrackCandidates'
   [11] Prefetching for module FastTrackerRecHitMaskProducer/'lowPtTripletStepMasks'
   [12] Prefetching for module TrackProducer/'highPtTripletStepTracks'
   [13] Prefetching for module TrackCandidateProducer/'highPtTripletStepTrackCandidates'
   [14] Prefetching for module FastTrackerRecHitMaskProducer/'highPtTripletStepMasks'
   [15] Prefetching for module TrackProducer/'lowPtQuadStepTracks'
   [16] Prefetching for module TrackCandidateProducer/'lowPtQuadStepTrackCandidates'
   [17] Prefetching for module FastTrackerRecHitMaskProducer/'lowPtQuadStepMasks'
   [18] Calling method for module TrackTfClassifier/'initialStep'
Exception Message:
No "TfGraphRecord" record found in the EventSetup.

 The Record is delivered by an ESSource or ESProducer but there is no valid IOV for the synchronization value.
 Please check 
   a) if the synchronization value is reasonable and report to the hypernews if it is not.
   b) else check that all ESSources have been properly configured. 
----- End Fatal Exception -------------------------------------------------

abdoulline · 2021-10-25T16:40:43Z

...So, waiting for a new IB to come out for test(s)...

slava77 · 2021-10-25T17:22:57Z

@cmsbuild please test

tvami · 2021-10-25T20:02:14Z

urgent

backport should go into 12_0_3_patch1

cmsbuild · 2021-10-25T20:23:05Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-986f2b/19928/summary.html
COMMIT: f190b38
CMSSW: CMSSW_12_1_X_2021-10-25-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35806/19928/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 68 differences found in the comparisons
DQMHistoTests: Total files compared: 41
DQMHistoTests: Total histograms compared: 2797338
DQMHistoTests: Total failures: 96
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2797220
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 40 files compared)
Checked 173 log files, 37 edm output root files, 41 DQM output files
TriggerResults: no differences found

missirol · 2021-10-25T21:25:30Z

+hlt

the change in RecoEgamma/EgammaHLTProducers does not affect any workflow
(and HLT already uses useM3 = False)
HLTEgPhaseIITestSequence_cff was added in #32519 only for testing purposes
(it could arguably be removed altogether once the HLT Phase-2 menu is integrated in master, see #35342)

missirol · 2021-10-25T21:38:37Z

Worth noting that in the current HLT Phase-2 menu there are a couple of cases where useM3 = True [1] [2].
FYI: @fwyzard @trtomei @swagata87 @khaosmos93 @pallabidas @SWuchterl.

perrotta · 2021-10-26T05:41:12Z

@slava77 this is identical to the backport PR #35807, that you already signed: I think this can also be (considered as) signed by reco, correct?

slava77 · 2021-10-26T12:40:53Z

+reconstruction

for #35806 f190b38

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show differences in run-3 and phase-2 workflows only in HCAL eaux variable, corresponding to the M3 energy calculation, which is not filled anymore.

cmsbuild · 2021-10-26T12:41:13Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-10-26T13:21:28Z

+1

M3 switched off for Run3

795dd3c

cmsbuild added this to the CMSSW_12_1_X milestone Oct 24, 2021

cmsbuild added code-checks-pending hlt-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 24, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 24, 2021

abdoulline mentioned this pull request Oct 24, 2021

[12_0_X] HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35807

Merged

abdoulline changed the title ~~HCAL: auxiliary M3 reconstruction fit switching off for Run3~~ HCAL: switch off auxiliary M3 reconstruction fit for Run3 Oct 24, 2021

cmsbuild added tests-started and removed tests-pending labels Oct 24, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 24, 2021

Martin-Grunewald reviewed Oct 25, 2021

View reviewed changes

restore HLTEgPhaseIITestSequence_cff.py

2b50725

cmsbuild added code-checks-pending tests-pending and removed hlt-pending tests-approved code-checks-approved labels Oct 25, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 25, 2021

cmsbuild added tests-rejected and removed tests-started labels Oct 25, 2021

cmsbuild added tests-started and removed tests-rejected labels Oct 25, 2021

cmsbuild added the urgent label Oct 25, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 25, 2021

cmsbuild added hlt-approved and removed hlt-pending labels Oct 25, 2021

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Oct 26, 2021

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2021

cmsbuild merged commit d8f8424 into cms-sw:master Oct 26, 2021

abdoulline deleted the M3_off_Run3 branch October 27, 2021 05:16

cmsbuild mentioned this pull request Oct 27, 2021

default initialize HiPuRhoProducer::postOrphan_ #35857

Merged

slava77 mentioned this pull request Nov 3, 2021

crash in HBHEPhase1Reconstructor in PromptReco_Run345915_ZeroBias19 #35785

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35806

HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35806

abdoulline commented Oct 24, 2021 •

edited

Loading

cmsbuild commented Oct 24, 2021

cmsbuild commented Oct 24, 2021

perrotta commented Oct 24, 2021

cmsbuild commented Oct 24, 2021

Martin-Grunewald Oct 25, 2021

abdoulline Oct 25, 2021

Martin-Grunewald Oct 25, 2021

Martin-Grunewald Oct 25, 2021

abdoulline Oct 25, 2021

Martin-Grunewald Oct 25, 2021

abdoulline Oct 25, 2021 •

edited

Loading

mariadalfonso Oct 25, 2021

abdoulline Oct 25, 2021 •

edited

Loading

cmsbuild commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

abdoulline commented Oct 25, 2021

slava77 commented Oct 25, 2021

tvami commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

missirol commented Oct 25, 2021

missirol commented Oct 25, 2021

perrotta commented Oct 26, 2021

slava77 commented Oct 26, 2021

cmsbuild commented Oct 26, 2021

perrotta commented Oct 26, 2021

HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35806

HCAL: switch off auxiliary M3 reconstruction fit for Run3 #35806

Conversation

abdoulline commented Oct 24, 2021 • edited Loading

PR description:

PR validation:

cmsbuild commented Oct 24, 2021

cmsbuild commented Oct 24, 2021

perrotta commented Oct 24, 2021

cmsbuild commented Oct 24, 2021

Comparison Summary

Martin-Grunewald Oct 25, 2021

Choose a reason for hiding this comment

abdoulline Oct 25, 2021

Choose a reason for hiding this comment

Martin-Grunewald Oct 25, 2021

Choose a reason for hiding this comment

Martin-Grunewald Oct 25, 2021

Choose a reason for hiding this comment

abdoulline Oct 25, 2021

Choose a reason for hiding this comment

Martin-Grunewald Oct 25, 2021

Choose a reason for hiding this comment

abdoulline Oct 25, 2021 • edited Loading

Choose a reason for hiding this comment

mariadalfonso Oct 25, 2021

Choose a reason for hiding this comment

abdoulline Oct 25, 2021 • edited Loading

Choose a reason for hiding this comment

cmsbuild commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

RelVals

abdoulline commented Oct 25, 2021

slava77 commented Oct 25, 2021

tvami commented Oct 25, 2021

cmsbuild commented Oct 25, 2021

Comparison Summary

missirol commented Oct 25, 2021

missirol commented Oct 25, 2021

perrotta commented Oct 26, 2021

slava77 commented Oct 26, 2021

cmsbuild commented Oct 26, 2021

perrotta commented Oct 26, 2021

abdoulline commented Oct 24, 2021 •

edited

Loading

abdoulline Oct 25, 2021 •

edited

Loading

abdoulline Oct 25, 2021 •

edited

Loading