Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic Reduction Network for photon energy regression in miniAOD using the SonicTriton service #37134

Merged
merged 42 commits into from
Apr 29, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
f3fee7c
One merged commit
ssrothman Mar 3, 2022
32c5854
Fix ES data getting to be compliant with newer CMSSW
ssrothman Mar 3, 2022
2da29a3
Added test
ssrothman Mar 3, 2022
616831b
code-format and code-checks
ssrothman Mar 3, 2022
3cdd06c
Code-format (again??)
ssrothman Mar 3, 2022
0bf9e99
Made test shorter
ssrothman Mar 7, 2022
da3c714
Fixed model name
ssrothman Mar 7, 2022
5af31a1
Not sure what's going on here tbh...
ssrothman Mar 7, 2022
fe1ee8e
Revert "Not sure what's going on here tbh..."
ssrothman Mar 7, 2022
c65a63d
Rename customisation function appropriately
ssrothman Mar 7, 2022
e36b116
Updated test config to call correct customisation function name
ssrothman Mar 7, 2022
26dfae6
Fixed customisation function to be compatible with cmsDriver
ssrothman Mar 8, 2022
30e291e
Almost working.....
ssrothman Mar 9, 2022
82ced8d
Kevin found the problem :D
ssrothman Mar 10, 2022
2f80e2f
Removed test
ssrothman Mar 11, 2022
d357aa7
UserFloat functionality
ssrothman Mar 16, 2022
9556a83
Fixed fatal typos
ssrothman Mar 16, 2022
2637202
No reason to touch runtests.sh
ssrothman Mar 16, 2022
1a5de2f
Pointers..
ssrothman Mar 16, 2022
f994140
Clean up DRNCorrectionProducerT
ssrothman Mar 16, 2022
e5db2ac
Error fixing...
ssrothman Mar 16, 2022
ec15ac7
Should compile now
ssrothman Mar 16, 2022
cc93d65
empty commit to get cms bot to rerun code checks
ssrothman Mar 16, 2022
c3927d0
Code-format
ssrothman Mar 16, 2022
722602b
Don't need ES Handles
ssrothman Mar 17, 2022
c658fab
No longer pointers
ssrothman Mar 17, 2022
509d4a1
Typo fix
ssrothman Mar 17, 2022
84c52f2
Adding process modifier
ssrothman Mar 17, 2022
0e55eb0
Typo fix
ssrothman Mar 17, 2022
c99da3c
userFloats by default
ssrothman Mar 17, 2022
8a69393
RelVal use modifier
ssrothman Mar 17, 2022
fa89ab1
Replace p4 matching
ssrothman Mar 17, 2022
bb7c727
Clean up after removing p4 comparison
ssrothman Mar 17, 2022
bccd830
Merge branch 'noHandle' into photon_rebase2
ssrothman Mar 17, 2022
8c0bd7c
Code-format
ssrothman Mar 17, 2022
1ab9afd
photonDRN modifier now includes enableSonicTriton
ssrothman Apr 4, 2022
fc9c472
use fillDescriptions defaults for recHits
ssrothman Apr 4, 2022
78558dc
shared memory on
ssrothman Apr 4, 2022
7ecb877
test clone...
ssrothman Apr 13, 2022
a8702fa
Revert "test clone..."
ssrothman Apr 14, 2022
5977c4b
Cloning fillDescriptions config
ssrothman Apr 14, 2022
c8484f9
Same for electrons
ssrothman Apr 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions PhysicsTools/PatAlgos/python/slimming/enableDRN.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import FWCore.ParameterSet.Config as cms
from PhysicsTools.PatAlgos.tools.helpers import getPatAlgosToolsTask

def enableDRN(process):
process.load("PhysicsTools.PatAlgos.slimming.patPhotonDRNCorrector_cfi")
process.DRNTask = cms.Task(process.patPhotonsDRN)
process.schedule.associate(process.DRNTask)
process.slimmedPhotons.modifierConfig.modifications.append(
cms.PSet( modifierName = cms.string("EGRegressionModifierDRN"),
patPhotons = cms.PSet(
source = cms.InputTag('selectedPatPhotons'),
correctionsSource = cms.InputTag('patPhotonsDRN')
)
)
)

return process
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import FWCore.ParameterSet.Config as cms

patElectronsDRN = cms.EDProducer("PatElectronDRNCorrectionProducer",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be better to clone the existing config from the one provided by fillDescriptions (I recall a similar comment on the previous DRN PR)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you just mean to use the fillDescriptions defaults for more parameters, or is there something more I should do here? I have updated it to use default values for the RecHit collections; should I add more defaults?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean using .clone to avoid specifying the config options (and the types) that are already known.
3cb2459#diff-2d4d8dba5a6cd14974d0d42a9896706879c369b9ca833acca0f8e9f1883b1500R3

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused about how to do this. If I try to import RecoEgamma.EgammaTools.PatPhotonDRNCorrectionProducer_cfi I get a ModuleNotFoundError. Where is this default config produced by fillDescriptions()? How (if at all) does it interact with class templating?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your working area, you can look at $CMSSW_BASE/cfipython to find the default configs.

To ensure distinct cfi files are generated for different instances of the same class template, see this example:

//to ensure distinct cfi names
descriptions.addWithDefaultLabel(desc);

particleSource = cms.InputTag("selectedPatElectrons"),
rhoName = cms.InputTag("fixedGridRhoFastjetAll"),
reducedEcalRecHitsEB = cms.InputTag("reducedEcalRecHitsEB"),
reducedEcalRecHitsEE = cms.InputTag("reducedEcalRecHitsEE"),
reducedEcalRecHitsES = cms.InputTag("reducedEcalRecHitsES"),

Client = cms.PSet(
mode = cms.string("Async"),
modelName = cms.string("electronObjectEnsemble"),
modelConfigPath = cms.FileInPath("RecoEgamma/EgammaElectronProducers/data/models/electronObjectEnsemble/config.pbtxt"),
ssrothman marked this conversation as resolved.
Show resolved Hide resolved
allowedTries = cms.untracked.uint32(1),
timeout = cms.untracked.uint32(10),
useSharedMemory = cms.untracked.bool(False),
verbose = cms.untracked.bool(True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is verbose=True intentional? I still have the feeling that each physics model having their own version of technical Triton/Sonic config values (e.g. timeout, shared mem, retries) is not optimal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, this is unavoidable. Different models have different execution times, therefore they cannot all share a single timeout. Some models might need to have shared memory disabled for a variety of reasons, etc.

However, I agree that verbose=True should be disabled in the PR (it's mainly for testing/profiling).

)
)
19 changes: 19 additions & 0 deletions PhysicsTools/PatAlgos/python/slimming/patPhotonDRNCorrector_cfi.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import FWCore.ParameterSet.Config as cms

patPhotonsDRN = cms.EDProducer("PatPhotonDRNCorrectionProducer",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment here regarding cloning the producer config

particleSource = cms.InputTag("selectedPatPhotons"),
rhoName = cms.InputTag("fixedGridRhoFastjetAll"),
reducedEcalRecHitsEB = cms.InputTag("reducedEcalRecHitsEB"),
reducedEcalRecHitsEE = cms.InputTag("reducedEcalRecHitsEE"),
reducedEcalRecHitsES = cms.InputTag("reducedEcalRecHitsES"),

Client = cms.PSet(
mode = cms.string("Async"),
modelName = cms.string("photonObjectBothEnsemble"),
modelConfigPath = cms.FileInPath("RecoEgamma/EgammaPhotonProducers/data/models/photonObjectEnsemble/config.pbtxt"),
ssrothman marked this conversation as resolved.
Show resolved Hide resolved
allowedTries = cms.untracked.uint32(1),
timeout = cms.untracked.uint32(10),
useSharedMemory = cms.untracked.bool(False),
verbose = cms.untracked.bool(True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment here.

)
)
269 changes: 269 additions & 0 deletions PhysicsTools/PatAlgos/test/DRNtest_Hgg.py

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions PhysicsTools/PatAlgos/test/runtests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,4 @@ cmsRun ${LOCAL_TEST_DIR}/patTuple_updateJets_fromMiniAOD_cfg.py || die 'Failure
# cmsRun ${LOCAL_TEST_DIR}/patTuple_onlyPhotons_cfg.py || die 'Failure using patTuple_onlyPhotons_cfg.py' $?
# cmsRun ${LOCAL_TEST_DIR}/patTuple_onlyTaus_cfg.py || die 'Failure using patTuple_onlyTaus_cfg.py' $?

cmsRun ${LOCAL_TEST_DIR}/DRNTest_cfg.py || die "Failure using DRNtest" $?
1 change: 1 addition & 0 deletions RecoEgamma/EgammaTools/BuildFile.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
<use name="RecoEcal/EgammaCoreTools"/>
<use name="DataFormats/ParticleFlowReco"/>
<use name="PhysicsTools/TensorFlow" />
<use name="HeterogeneousCore/SonicTriton"/>
<export>
<lib name="1"/>
</export>
Loading