features for UL re-miniAOD in 10_6_X

[slava77]

This is to keep track of features needed for the UL re-miniAOD.
The 2017 will likely happen first.
The target start date is still unclear (it will probably be around October 2019).

As of Aug 7 (after #31072 was merged) the re-miniAOD definitions aiming for UL re-miniAOD should use a run2_miniAOD_UL modifier, if the configuration is to be controlled by a modifier.

[as of Dec 9]

• Disabled PCA inputs for deepTau v2 (10_6_X) #27882 "Disabled PCA inputs for deepTau v2". Target: all re-miniAOD. (merged Sep 19, 2019)
  • the change to deepTau2017v2p1 is already applied in UL 2016 and 2018 by default
  • controlled by run2_miniAOD_devel
• Add customizer for PUPPI tune v13 [10_6_X backport] #28845 "Add customizer for PUPPI tune v13" (merged Feb 27, 2020)
  • this is implemented with a customize method; superseded by v14
• PUPPI tune v14 [10_6_X] #29601 PUPPI tune v14 (merged May 14, 2020)
• Adding parameters needed for PUjetID 94X and 102X (backport to 106X) #29045 Adding parameters needed for PUjetID 94X and 102X (merged Mar 3, 2020)
  • bugfix for run2_miniAOD_devel -based setup in fix miniaod_devel modifier use for pileupJetId.algos #29377 (merged April 2, 2020)
  • controlled by run2_miniAOD_devel
• Fix MET unclustered energy computation [10_6_X] #29330 Fix MET unclustered energy computation (merged April 1, 2020)
  • controlled by run2_miniAOD_devel
• Fix MET Significance, Covariance Matrix and unclustered energy lepton subtraction [10_6_X] #29392 Fix MET Significance, Covariance Matrix and unclustered energy lepton subtraction (merged Apr 21, 2020)
  • controlled by run2_miniAOD_devel
• keeping METsumPtUnclustered (backport of #28370) #29521 Keeping sumPt from MET Significance calculation (merged Apr 21, 2020)
  • enabled, no era modifiers
• [10_6_X] MXNet-based implementation of the ParticleNet tagger #29062 MXNet-based implementation of the ParticleNet tagger (merged Apr 18, 2020)
  • controlled by run2_miniAOD_devel
• Backport of inverse beta variable in pat::Muons for 10_6_X #29324 inverse beta variable in pat::Muons (merged Apr 17, 2020)
  • enabled, no era modifiers
• Update of the lostTrack collection for MUO (back-port of #29389) #29529 Update of the lostTrack collection for MUO (merged Apr 23, 2020)
• Backport of #28753 (fix the sign of pat::Muon dxy to take the one from the track) #29537 fix sign of pat::Muon::dxy (merged May 1, 2020)
• PPS: updates for re-miniAOD (backport of #29770) #29771 PPS rereco-in-miniAOD (merged May 25, 2020)
  • controlled by run2_miniAOD_devel
• Restructured code of againstElectronDeadECAL tauID (10_6_X) #29739 Tau againstElectronDeadECAL (merged May 27, 2020 )
  • controlled by run2_miniAOD_devel
• Fix bug affecting boosted tau isolation (10_6_X) #30066 Fix bug affecting boosted tau isolation (merged June 4, 2020)
• Backport of UL scales and smearings to 106X  #30199 Updates to electron/photon scales and smearings for UL (merged June 19, 2020)
• [10.6] Backport onnxrt 1.3.0 to 10_6 cmsdist#5902 update ONNXRuntime to 1.3.0 (merged June 17, 2020)
• [10_6_X] ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX #30123 ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX (technical; significant speedup) (merged July 2, 2020)
• baseline workflows for UL reminiAOD #30352 define UL re-miniAOD workflows; technical at this point, repeats UL main miniAOD (merged June 26, 2020)
• [10_6_X] feat: Introduce DeepDoubleX V2 #30542 deepDoubleX V2; capability only, plan to enable later (merged July 13, 2020)
• ONNX-based Higgs to bb Interaction Network Tagger [10_6_X] #30584 Hbb Interaction network tagger (merged July 20, 2020)
• 106X backport of #30738 (Use dxy sign of GSF track instead of IPTools for pat::Electron) #30892 Use dxy sign of GSF track instead of IPTools for pat::Electron (merged Aug 4, 2020)
• 106X backport of: Pile-Up ID training and WPs for 2017 UL samples #30935 Pile-Up ID training and WPs for 2017 UL samples (merged Aug 4, 2020)
• migrate from run2_miniAOD_devel to run2_miniAOD_UL for miniAOD setup UL workflows #31072 migrate run2_miniAOD_devel parts suitable for UL re-mini to use run2_miniAOD_UL (merged Aug 7, 2020)
• Fix to enable hpsPFTauDiscriminationByDeadECALElectronRejection in run2 reminiAODs (10_6_X) #31044 fix to enable hpsPFTauDiscriminationByDeadECALElectronRejection in run2 reminiAOD (merged Aug 16, 2020)
• [10_6_X] Use ONNXRuntime for ParticleNet and update the training for UL #31037 updated training for ParticleNet tagger (merged Aug 17, 2020)
• Adapt track-vertex association in PUPPI (v15) [10_6_X] #31214 Adapt track-vertex association in PUPPI (v15) (merged Aug 25, 2020)
• Add DeepMET into NanoAOD for UltraLegacy in 10_6_X (backport of 30291 and 30926) #31120 deepMet for nanoAOD, includes an operational bugfix in DeepMETProducer (merged Aug 25, 2020)
• addition of new MET filter BadPFMuonDzFilter_cfi.py in CMSSW_10_6_X : i.e. backport from 11_1_X of PR 30015 and 30974 #31087 bad PF muon filter (merged Sep 14, 2020)
• Update tau anti electron discriminators in RECO and MiniAOD (10_6_X) #31065 Update tau anti electron discriminators (merged Sep 1, 2020)
• [10_6_X] improving performance of PuppiProducer #31290 improving (computational) performance of PuppiProducer (merged Sep 4, 2020)
• HF Noise Event Filter 106X backport #31413 HF Noise Event Filter (merged Sep 16, 2020)
• HF jet shower shape (106X backport) #31420 HF jet shower shape (merged Sep 18, 2020)
• 10_6_X backport of "HCAL: slimmed collections... for miniAOD" (PR #31375) #31510 HCAL slimmed collections (merged Sep 23, 2020)
• [10_6_X] ParticleNetAK4 jet tagger #31615 ParticleNet AK4 jet tagger (merged Oct 7, 2020)
• [10_6_X] DDXv2 and BTV changes in Nano #31618, #31786 #31847 Activate deepDoubleXV2 (merged Nov 4, 2020)
• [RecoEgamma/EgammaTools] Backport of UL 2018 scale and smearing corrections #31995 Electron Scale and smearing for UL (merged Nov 4, 2020)
• speedup: switch EgammaTools EnergyScaleCorrection scales and smearings to use std::map (backport of #32053) #32055 speedup at job start in EnergyScaleCorrection (merged Nov 10, 2020)
• Expand track information in lostTracks collection (10_6_X) #32303 Expand track information in lostTracks collection, for tau reco-from-mini (merged Nov 30, 2020)
• Pr106 L1T Unprefireable Bit Functionality for miniAOD Reprocessing #31984 "unprefireable bit info" (merged Dec 2, 2020)
• Add low-pT electrons to MINIAOD, update ID, improve end user experience (back port of 31220) #32372 update LowPtElectrons in bParking config and add low-pt electrons in default miniAOD (merged Dec 9, 2020)

The following features were descoped:

• (no 10_6_X PR) LowPtElectrons: final energy regression and ID #32391 energy regression for low-pt electrons, including updated training of IDs (all can be redone from miniAOD, not in LowPtElectrons: final energy regression and ID #32391 as of Dec 8)
• quark gluon likelihood retraining (controlled by GT; possibly descoped features for UL re-miniAOD in 10_6_X #27889 (comment))
• update of the prefiring maps. (will update a file in https://github.com/cms-data/PhysicsTools-PatUtils; possibly descoped features for UL re-miniAOD in 10_6_X #27889 (comment))

A bit outside of re-miniAOD scope (requires fresh MC/rereco)

• Store TrackExtra, TrackingRecHits, and SiPixel/StripClusters for muons in AOD+MINIAOD (10_6_X) #31350 Store TrackExtra, TrackingRecHits, and SiPixel/StripClusters for muons in AOD+MINIAOD

[slava77]

assign reconstruction,xpog,operations

[cmsbuild]

New categories assigned: operations,xpog,reconstruction

@slava77,@fabiocos,@franzoni,@fgolf,@perrotta,@davidlange6,@kpedro88,@peruzzim you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Issue was created by @slava77 Slava Krutelyov.

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos, @kpedro88 can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[slava77]

it sounds like there is still time for deploying the fix in the 2018 UL.
I propose to make run2_tau_ul_2018 and run2_tau_ul_2016 modifiers which can be added to Run2_2018 and Run2_2016 eras.
There is not need to propagate these to the master.

@fabiocos @kpedro88 @srimanob this is as a follow up to the discussion today in the ORP.
Please check if this proposal makes sense to you as well.

[peruzzim]

it sounds like there is still time for deploying the fix in the 2018 UL.
I propose to make run2_tau_ul_2018 and run2_tau_ul_2016 modifiers which can be added to Run2_2018 and Run2_2016 eras.
There is not need to propagate these to the master.

@fabiocos @kpedro88 @srimanob this is as a follow up to the discussion today in the ORP.
Please check if this proposal makes sense to you as well.

I agree with the above, but please let's keep it active also under the generic devel modifier, so we will remember to add it also to 2017 in a future re-miniAOD of that.

[peruzzim]

taking note of #27929 (comment)

[peruzzim]

taking note of #28844

[peruzzim]

taking note of #25884

[peruzzim]

also taking note of missing unprefireable bit information in UL miniAOD

The envisaged solution is:

• run something equivalent to the L1Repack:uGT step, re-emulating ExternalAlgBlock in data with a dedicated customisation
• adapt NanoAOD configuration accordingly

@rekovic @srimanob

[peruzzim]

assign l1

[cmsbuild]

New categories assigned: l1

@benkrikler,@rekovic you have been requested to review this Pull request/Issue and eventually sign? Thanks

[srimanob]

FYI @ameyer-cern

[peruzzim]

taking note of MUO request for additional tracks to be added in MiniAOD supporting SF derivation without access to AOD

@dntaylor

[slava77]

@danbarto @ahinzmann
I suppose that we also need a backport for #28370

[ahinzmann]

Will need also #29392

[peruzzim]

taking note of #29062

[slava77]

I've updated the issue description #27889 (comment) with the list of features mentioned in the thread.
Please check if something is missing

[slava77]

I missed the PPS plans; added now in the issue description

@jan-kaspar @fabferro

[slava77]

updates from Oct 2:

• 10_6_X backport of "HCAL: slimmed collections... for miniAOD" (PR #31375) #31510 HCAL slimmed collections was merged Sep 23, 2020)
• new feature added: [10_6_X] ParticleNetAK4 jet tagger #31615 ParticleNet AK4 jet tagger
• info updated with master PR now available ( feat: switch to DDXv2 as baseline #31618) for Activate deepDoubleXV2

[slava77]

updates from Oct 2:

* #31510 HCAL slimmed collections  was merged Sep 23, 2020)

* new feature added: #31615 ParticleNet AK4 jet tagger

* info updated with master PR now available ( #31618) for Activate deepDoubleXV2

also, added a note for "produce lowPtGsfEle slimmed collections": Target by October 10 or so.

[slava77]

updates

• [10_6_X] ParticleNetAK4 jet tagger #31615 ParticleNet AK4 jet tagger (merged Oct 7, 2020)

added

• (backport needed for) Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220 update LowPtElectrons in bParking config
  • this is a part of resolution to produce lowPtGsfEle slimmed collections for the default miniAOD #25884

[mariadalfonso]

update

• Electron Scale and smearing for UL
  (backport needed for) UL2018 scale and smearing added #31936

[slava77]

update

* Electron Scale and smearing for UL
  (backport needed for) #31936

this is added now in the description

also updated at the top:

• [10_6_X] DDXv2 and BTV changes in Nano #31618, #31786 #31847 Activate deepDoubleXV2 (now a 10_6_X PR)
• Pr106 L1T Unprefireable Bit Functionality for miniAOD Reprocessing #31984 "unprefireable bit info"
  • only the feature is in, customization L1TStage2SetPrefireVetoBit needs to be propagated to the workflows
• Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220 update LowPtElectrons ... : the item was merged with produce lowPtGsfEle slimmed collections for the default miniAOD #25884 now that Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220 includes the low-pt elements for the default miniAOD

[slava77]

updates:

• now merged
  • [10_6_X] DDXv2 and BTV changes in Nano #31618, #31786 #31847 Activate deepDoubleXV2 (merged Nov 4, 2020)
  • [RecoEgamma/EgammaTools] Backport of UL 2018 scale and smearing corrections #31995 Electron Scale and smearing for UL (merged Nov 4, 2020)
  • speedup: switch EgammaTools EnergyScaleCorrection scales and smearings to use std::map (backport of #32053) #32055 speedup at job start in EnergyScaleCorrection (merged Nov 10, 2020)

[slava77]

@lathomas @ahinzmann @kirschen @cms-sw/xpog-l2
please comment on the status of the following items which do not yet have a PR :

• quark gluon likelihood retraining
• update of the prefiring maps. (this one would just be an update of the related file in https://github.com/cms-data/PhysicsTools-PatUtils in principle)

[lathomas]

Both are ongoing. 2017 prefiring maps could be ready soon. 2016 will take longer (need UL16 JECs as inputs).
For qgl, it might take some time. This would be nice to have in reMINIAOD but not critical (i.e. I wouldn't delay everything for this).

[slava77]

Both are ongoing. 2017 prefiring maps could be ready soon. 2016 will take longer (need UL16 JECs as inputs).
For qgl, it might take some time. This would be nice to have in reMINIAOD but not critical (i.e. I wouldn't delay everything for this).

Thank you for clarifying.
Do I read your comment correctly that the prefiring maps are still a required feature for remini,
while the qgl is more of a nice to have?

[lathomas]

I don't think that the prefiring maps are such a big deal than they justify delaying reMINIAOD, given that the prefiring weights can be recomputed on top of MINIAOD.

That being said, the timeline is the following:
2017 maps should hopefully come soon (in a week or two).
2016 maps won't be ready before JECs so it will probably take a month (but maybe reMINI of UL16 MC won't have started yet?).

By the way, the prefiring weights are NOT saved in MINIAOD right now. The producer is however used to compute the weights that are stored NANOAOD, and this is why we would care (but then a future reNANO could add them)

[slava77]

@bainbrid
please list the remaining features for low-pt electrons that will not enter #31220 and are planned for separate PRs

[bainbrid]

In response to #27889 (comment):

For low-pT electrons:

• we confirm that we will add code/configuration to perform an energy regression using RECO/AOD inputs
• we confirm the need for a final ID training (to replace the current placeholder) and associated PR for 11_2 and 10_6.
• we are considering to embed flags that identify if the pat::Electron originates from a converted photon
• we are considering to allow updates to the regression and ID using MINIAOD inputs ("MiniAODfromMiniAOD")

@crovelli @cavalari @nancymarinelli

[perrotta]

Additional tracks information for the lostTracks, which was submitted by the tau group in the master with #32242, could be considered for the backport, if it can be done in the shadow of the other pending pull requests (see #32242 (comment))

[perrotta]

In response to #27889 (comment):

For low-pT electrons:

• we confirm that we will add code/configuration to perform an energy regression using RECO/AOD inputs
• we confirm the need for a final ID training (to replace the current placeholder) and associated PR for 11_2 and 10_6.
• we are considering to embed flags that identify if the pat::Electron originates from a converted photon
• we are considering to allow updates to the regression and ID using MINIAOD inputs ("MiniAODfromMiniAOD")

@crovelli @fcavallo @nancymarinelli

@bainbrid I imagine you were supposed to tag Francesca Cavallari (of whom I don't know the github name, sorry...) instead of Francesca Cavallo (Muon DPG)

[bainbrid]

@crovelli @fcavallo @nancymarinelli

@bainbrid I imagine you were supposed to tag Francesca Cavallari (of whom I don't know the github name, sorry...) instead of Francesca Cavallo (Muon DPG)

Fixed, thanks! @cavalari

[slava77]

updates from today (now in the issue description/list):

• added
  • Expand track information in lostTracks collection (10_6_X) #32303 Expand track information in lostTracks collection, for tau reco-from-mini
  • (no PR/PRs) energy regression for low-pt electrons, including updated training of IDs (all can be redone from miniAOD, can be descoped)
• status/description updated
  • Pr106 L1T Unprefireable Bit Functionality for miniAOD Reprocessing #31984 "unprefireable bit info" (fully signed Nov 26, 2020)
  • (backport needed for) Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220 update LowPtElectrons in bParking config and add low-pt electrons in default miniAOD (signed by reco Nov 26, 2020)
  • quark gluon likelihood retraining (controlled by GT; possibly descoped features for UL re-miniAOD in 10_6_X #27889 (comment))
  • update of the prefiring maps. (will update a file in https://github.com/cms-data/PhysicsTools-PatUtils; possibly descoped features for UL re-miniAOD in 10_6_X #27889 (comment))

[slava77]

updates from near the finish line :

• merged:
  • Expand track information in lostTracks collection (10_6_X) #32303 Expand track information in lostTracks collection, for tau reco-from-mini (merged Nov 30, 2020)
  • Pr106 L1T Unprefireable Bit Functionality for miniAOD Reprocessing #31984 "unprefireable bit info" (merged Dec 2, 2020)
• pending, but almost ready
  • Add low-pT electrons to MINIAOD, update ID, improve end user experience (back port of 31220) #32372 update LowPtElectrons in bParking config and add low-pt electrons in default miniAOD (signed by reco Dec 8, 2020)
• remaining features were descoped (all expected to be re-doable from miniAOD inputs, which makes these candidates for the next nanoAOD)
  • low-pt electron energy regression (controlled by GT, but also needs code updates) and 2020Nov28 ID
  • quark gluon likelihood retraining (controlled by GT)
  • update of the prefiring maps

[slava77]

+1

it sounds like we are in a production stage now

[mariadalfonso]

+xpog

indeed production advancing

[srimanob]

@cms-sw/operations-l2 should this issue be closed?

[perrotta]

@cms-sw/operations-l2 should this issue be closed?

I think so