Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805

CeliaFernandez · 2022-05-04T18:14:19Z

PR description:

This PR is a follow-up from a previous one: #36926

It adds a new reco::Muon collection, called 'displacedMuons', that includes the dedicated displaced muon reconstructions: 'displacedTracks', 'displacedGlobalMuons' and 'displacedStandAloneMuons' (all of them saved in AOD as reco::Track collections).

In this PR we add the necessary modifications to:

Cross-clean the existing reco::Track objects in order to avoid the duplication or loss of displaced muon candidates in the output collection.
Compute the main muon variables, including isolation, time, recoTracksExtras... etc
Save this information in AOD and RECO, as configured for standard 'muons' collection
Read the new reco::Muon collection and create a pat::Muon collection to be saved in MiniAOD
Add a filter to preselect the muons stored in the new pat::Muon collection and reduce the size increase in the new samples.

The sequences are implemented to run independently to the default reco::Muon 'muons' collection and avoid interferences. Muons of the 'muons' collection should remain untouchable. The sequences also emulate the existing implementation and make use of the existing tools and modules so the new information is computed with the default parameters configuration.

The following packages are modified:

RecoMuon/Configuration
RecoMuon/Identification
RecoMuon/MuonIsolationProducers
RecoMuon/MuonIsolation
PhysicsTools/PatAlgos

This new collection was presented several times in CMS meetings. Further information can be found in:

https://indico.cern.ch/event/1077850/
https://indico.cern.ch/event/1122330/
https://indico.cern.ch/event/1128824/
https://indico.cern.ch/event/1134932/
https://indico.cern.ch/event/1153168/ (last report in Muon HLT&RECO meeting)

The increase in size is measured in ttbar (PU) simulation [11834.21 workflow] and totals +1% in AOD and +2.5% in MiniAOD.

Updates from the last presentation:
The filter was further optimized and the preselection requirements on muons were updated.

The implementation was built on top of CMSSW_12_3_0_pre2

@gkaratha @sscruz @trocino

PR validation:

We have checked out all dependencies and have a clean build. Code-checks were applied. Most runTheMatrix tests were successfully run as well:
runall-report-step123-.log

Studies about the physics performance are available in the presentations attached before.

…splacedStandAloneMuons to create a recoMuon collection with MuonReducedTrackExtras

…CO now but not in AOD, remove unecessary producers.

…onstruction is not available there

… are different there

…saved inside the pat muon

…pply filter to DTR

…from eras modifiers

cmsbuild · 2022-05-04T18:20:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37805/29723

This PR adds an extra 128KB to repository
Found files with invalid states:
- RecoMuon/MuonIdentification/python/filteredDisplacedMuons1stStep_cfi.py:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
- RecoMuon/MuonIdentification/plugins/DisplacedMuonFilterProducer.cc:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
- RecoMuon/MuonIdentification/plugins/DisplacedMuonFilterProducer.h:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167, Adding JPTjets to MiniAOD #36335
- File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): Adding JPTjets to MiniAOD #36335
- File RecoMuon/Configuration/python/RecoMuonPPonly_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/Configuration/python/RecoMuon_EventContent_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/Configuration/python/RecoMuon_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIdentification/python/displacedMuonReducedTrackExtras_cfi.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIdentification/python/displacedMuons_cfi.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolation/python/displacedMuonPFIsolationDeposits_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolation/python/displacedMuonPFIsolationValues_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolation/python/muonPFIsolation_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolationProducers/python/muIsoDepositCopies_cfi.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolationProducers/python/muIsoDeposits_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolationProducers/python/muIsolation_EventContent_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926
- File RecoMuon/MuonIsolationProducers/python/muIsolation_cff.py modified in PR(s): Add new reco::Muon 'displacedMuons' collection to AOD #36926

cmsbuild · 2022-05-04T18:21:12Z

A new Pull Request was created by @CeliaFernandez (Celia Fernández Madrazo) for master.

It involves the following packages:

PhysicsTools/PatAlgos (reconstruction)
RecoMuon/Configuration (reconstruction)
RecoMuon/MuonIdentification (reconstruction)
RecoMuon/MuonIsolation (reconstruction)
RecoMuon/MuonIsolationProducers (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @abbiendi, @emilbols, @mbluj, @demuller, @cericeci, @mmarionncern, @ahinzmann, @jhgoh, @jdolen, @HuguesBrun, @azotz, @seemasharmafnal, @trocino, @schoef, @rociovilar, @sscruz, @jdamgov, @bellan, @nhanvtran, @gkasieczka, @hatakeyamak, @andrzejnovak, @AlexDeMoor, @Fedespring, @JyothsnaKomaragiri, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jpata · 2022-05-05T09:34:51Z

@cmsbuild please test

jpata · 2022-05-05T09:34:56Z

type muon

cmsbuild · 2022-05-13T21:46:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37805/30007

This PR adds an extra 156KB to repository
Found files with invalid states:
- RecoMuon/MuonIdentification/python/filteredDisplacedMuons1stStep_cfi.py:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
- RecoMuon/MuonIdentification/plugins/DisplacedMuonFilterProducer.cc:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
- RecoMuon/MuonIdentification/plugins/DisplacedMuonFilterProducer.h:
  - Added: bcf92d5
  - Deleted: 4e2b9a7
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py modified in PR(s): Refactoring gen weight storage in EDM + Nano integration #32167, Adding JPTjets to MiniAOD #36335
- File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): Adding JPTjets to MiniAOD #36335

cmsbuild · 2022-05-13T21:47:23Z

Pull request #37805 was updated. @gouskos, @clacaputo, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso can you please check and sign again.

perrotta · 2022-05-13T21:50:17Z

please test
(I implemented the code-format directly here, to speed up)

CeliaFernandez · 2022-05-13T22:27:42Z

please test (I implemented the code-format directly here, to speed up)

Thanks @perrotta

cmsbuild · 2022-05-14T06:00:45Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-152438/24718/summary.html
COMMIT: 27c8a99
CMSSW: CMSSW_12_4_X_2022-05-13-1100/slc7_amd64_gcc10
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37805/24718/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3697377
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3697341
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 206 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

clacaputo · 2022-05-14T08:36:04Z

+reconstruction

re-sign Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805 (comment)
new reco::Muon displacedMuons collection added to AOD and MINIAOD
- The increase in size is measured in ttbar (PU) simulation [11834.21 workflow]:
  - +1% in AOD
  - +2.5% in MiniAOD
- Discussions with Muon POG and EXO PAG guarantee that the new collection size is minimal as it can be, without jeopardizing future physics analysis
- A review of the content will be made by the end of 2022
Profiling jobs don't show anomalous changes in CPU/MEM usage
No reco changes have been observed or are expected in other reco quantities

mariadalfonso · 2022-05-14T09:13:28Z

+xpog

(only code formats changes since the last sign-off)

cmsbuild · 2022-05-14T09:13:51Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-05-14T11:24:14Z

+1

qliphy · 2022-05-15T12:02:47Z

There is a unit test failure in IB after this PR gets merged.
@CeliaFernandez Would you please have a check asap? We are about to build 12_4_0_pre4.

https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc10/CMSSW_12_4_X_2022-05-15-0000/unitTestLogs/TauAnalysis/MCEmbeddingTools#/641-641

----- Begin Fatal Exception 15-May-2022 12:20:13 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
[0] Processing Event run: 274199 lumi: 125 event: 245779397 stream: 0
[1] Running path 'MINIAODSIMoutput_step'
[2] Prefetching for module PoolOutputModule/'MINIAODSIMoutput'
[3] Prefetching for module PATMuonSlimmer/'slimmedDisplacedMuons'
[4] Prefetching for module PATMuonSelector/'selectedPatDisplacedMuons'
[5] Prefetching for module PATMuonProducer/'patDisplacedMuons'
[6] Prefetching for module DisplacedMuonFilterProducer/'filteredDisplacedMuons'
[7] Prefetching for module MuonProducer/'displacedMuons'
[8] Prefetching for module CandIsolatorFromDeposits/'dispMuPFMeanDRIsoValuePU03'
[9] Calling method for module CandIsoDepositProducer/'dispMuPFIsoDepositPU'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Candidate
Looking for module label: displacedMuons1stStep
Looking for productInstanceName:

Additional Info:
[a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

perrotta · 2022-05-15T14:00:19Z

@qliphy I made an attempt fix in #37953
I let bot test it, to speed up things. Of course if anyone wants to test it, or propose more appropriate fixes, that would be welcome.

CeliaFernandez added 14 commits January 31, 2022 12:53

Add sequences to process displacedTracks, displacedGlobalMuons and di…

e90cc2e

…splacedStandAloneMuons to create a recoMuon collection with MuonReducedTrackExtras

Modify detector based isolation processing: Deposits only saved in RE…

25764ce

…CO now but not in AOD, remove unecessary producers.

Remove displacedMuons sequences from fastsim since displacedTrack rec…

0c974b0

…onstruction is not available there

Remove outdated lines

0e2664c

Do not use clusters in displacedMuonReducedTrackExtras since clusters…

4d2de56

… are different there

Add displacedMuons to miniAOD

25022c0

Add displacedMuon filter

bcf92d5

Set embedHighLevelSelection to True so impact parameter info is also …

e7546c9

…saved inside the pat muon

Remove filter from RECO step

4e2b9a7

Add filter to PAT displacedMuons processing

8dd921b

Adjust filter parameters and add selection of PAT displacedMuons

a79d06d

Modify displacedMuon filter parameters to keep good DSA and DGL and a…

ed93959

…pply filter to DTR

Update the filter to last version and remove the displaced sequences …

09287e0

…from eras modifiers

Code quality checks on new filter

ded4582

cmsbuild added this to the CMSSW_12_4_X milestone May 4, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 4, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 4, 2022

cmsbuild added muon tests-started and removed tests-pending labels May 5, 2022

jpata mentioned this pull request May 5, 2022

Add new reco::Muon 'displacedMuons' collection to AOD #36926

Closed

cmsbuild added code-checks-pending and removed code-checks-rejected labels May 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels May 13, 2022

cmsbuild added tests-started and removed tests-pending labels May 13, 2022

cmsbuild added tests-approved and removed tests-started labels May 14, 2022

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 14, 2022

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels May 14, 2022

cmsbuild added orp-approved and removed orp-pending labels May 14, 2022

cmsbuild merged commit e0e5d86 into cms-sw:master May 14, 2022

perrotta mentioned this pull request May 15, 2022

Add another excluded era for displaced muons #37953

Merged

slava77 mentioned this pull request May 17, 2022

reco comparisons: updated maps and monitoring for new products cms-sw/cms-bot#1762

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805

Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805

CeliaFernandez commented May 4, 2022

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

jpata commented May 5, 2022

jpata commented May 5, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

perrotta commented May 13, 2022

CeliaFernandez commented May 13, 2022

cmsbuild commented May 14, 2022

clacaputo commented May 14, 2022

mariadalfonso commented May 14, 2022

cmsbuild commented May 14, 2022

perrotta commented May 14, 2022

qliphy commented May 15, 2022

perrotta commented May 15, 2022

Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805

Add new reco::Muon 'displacedMuons' collection to AOD and MiniAOD #37805

Conversation

CeliaFernandez commented May 4, 2022

PR description:

PR validation:

cmsbuild commented May 4, 2022

cmsbuild commented May 4, 2022

jpata commented May 5, 2022

jpata commented May 5, 2022

cmsbuild commented May 13, 2022

cmsbuild commented May 13, 2022

perrotta commented May 13, 2022

CeliaFernandez commented May 13, 2022

cmsbuild commented May 14, 2022

Comparison Summary

clacaputo commented May 14, 2022

mariadalfonso commented May 14, 2022

cmsbuild commented May 14, 2022

perrotta commented May 14, 2022

qliphy commented May 15, 2022

perrotta commented May 15, 2022