Tracks with ill-defined momenta within pat::PackedCands

[mbluj]

In miniAOD samples (recent 94X/102X productions) there are rare tracks within pat:PackedCandidates (mainly lostTracks collection) with ill-defined momenta, i.e. pt being inf and/or eta nan.

The issue was discovered during big validation of tauReco@Mini and caused its failure. The failures of tauReco@Mini are fixed by protection introduced by #31582, but it was not checked if the ill-defined momenta are feature introduced in creation of pat::PackedCandidates (it can be caused either by compression or forcing a given mass of a candidate) or already original reco::Tracks are affected => probably should be followed up by XPOG.

An example of problematic pat::PackedCandidate can be found in /store/mc/RunIIAutumn18MiniAOD/TauGun_Pt-15to3000_13TeV_pythia8/MINIAODSIM/102X_upgrade2018_realistic_v15-v1/120000/1C5E6BED-FAFA-BC49-AC3C-DDBCC35CBE24.root, Run 1, Event 1556024, LumiSection 1557

[cmsbuild]

A new Issue was created by @mbluj .

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

assign xpog

[cmsbuild]

New categories assigned: xpog

@fgolf,@mariadalfonso,@gouskos,@peruzzim you have been requested to review this Pull request/Issue and eventually sign? Thanks

[mariadalfonso]

@bendavid @hatakeyamak @mtosi
Can you have a look at the source of the infinite ?

[hatakeyamak]

@mariadalfonso
Is it clear if this is packedCandidate issue? or reco pfcandidate issue?
If the former, it's more in the xpog area.

[slava77]

In miniAOD samples (recent 94X/102X productions) there are rare tracks within pat:PackedCandidates (mainly lostTracks collection) with ill-defined momenta, i.e. pt being inf and/or eta nan.

from #31582 (comment)

This looks like something that was fixed in 106X/master in #28043 / #27988

[slava77]

I think that this issue is a duplicate of #27374

[mbluj]

@mariadalfonso
Is it clear if this is packedCandidate issue? or reco pfcandidate issue?
If the former, it's more in the xpog area.

It is unlikely a PFCandidate issue as it looks be related to packedCandidates built on top of tracks unused by PFlow (lostTracks). It can be either issue of tracks (with a low quality) or a way in which the tracks are packed into packedCandidate dataformat.

To understand origin of the problem one should check original AOD events used to produce MiniAOD ones with problematic cands/tracks (I had not time to check, the original AODs are only on types). Probably the issue is fixed by #27988, but not checked explicitly.

[hatakeyamak]

I see. Thank you.
Perhaps, tau pog doesn't find this ill-behavior in 106x (or later) equivalent sample, we can consider this is solved?

[mbluj]

I see. Thank you.
Perhaps, tau pog doesn't find this ill-behavior in 106x (or later) equivalent sample, we can consider this is solved?

Hmm, I am afraid that there is not any equivalent 106X or later sample. I also do not know if big UL MiniAOD samples were produced with or without the fix (in small samples as relvals the issue can be not present because of its rarity). Really, the best thing will be to get the original AOD event (*) check the problematic track and eventually build miniAOD on top of it using 106X with the fix.

(*) BTW, do you know if edmPickEvent should work as it is failing for me.

[hatakeyamak]

What is the AOD sample? If it's on tape, we may have to do something extra.

[mbluj]

It is /TauGun_Pt-15to3000_13TeV_pythia8/RunIIAutumn18DRPremix-102X_upgrade2018_realistic_v15-v1/AODSIM on tapes accordingly to DAS.
I would not stage out the whole sample as it contains ~10M events. Instead, I would localize the block or even the file containing the event of interest, but as mentioned I was not able with edmPickEvents.py as it failed and have not had time to investigate further.

[hatakeyamak]

Do you know the specific file or block containing this event? If so, I can ask FNAL (T1_US_FNAL) computing folks.

[mbluj]

Unfortunately no. As written above, in principle it can be checked with edmPickEvents.py, but it fails. Namely, I tried:

edmPickEvents.py /TauGun_Pt-15to3000_13TeV_pythia8/RunIIAutumn18DRPremix-102X_upgrade2018_realistic_v15-v1/AODSIM 1:1557:1556024

as advised in https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookPickEvent, but it apparently has problems.

[hatakeyamak]

The relevant file should be on its way to disk to enable pickEvents.

[mbluj]

OK, great, thanks! Please let me know when it is there. Then, I will try to perform relevant checks.

[hatakeyamak]

Sounds good.

pickEvent still doesn't work for me, but
root://kodiak-se.baylor.edu//store/mc/RunIIAutumn18DRPremix/TauGun_Pt-15to3000_13TeV_pythia8/AODSIM/102X_upgrade2018_realistic_v15-v1/120000/5CE6FB17-A5FB-6843-86F7-C9C276A6AF16.root
should include this event.

edmFileUtil -e root://kodiak-se.baylor.edu//store/mc/RunIIAutumn18DRPremix/TauGun_Pt-15to3000_13TeV_pythia8/AODSIM/102X_upgrade2018_realistic_v15-v1/120000/5CE6FB17-A5FB-6843-86F7-C9C276A6AF16.root | grep 1556024
1 1557 1556024 39

Can you take it from here?

[mbluj]

Yes, thanks. I copied the file locally.

[mbluj]

I performed the checks with the following results:

1. In the event causing problems there is a track with pt=114263 which is above the maximal value allowed by the packing algorithm used by packedCands.
2. In 102X it gives in result a packedCandidate (in the lostTracks collection) with infinite pt (nan), but correct eta and phi. Then, a track retrieved from this packedCandidate (with bestTrack() method) receives not only undefined pt, but also eta, while phi value is finite but different than original one.
3. Backport of #27988 (Truncate pt of PackedCandidate) #28043 / Truncate pt of PackedCandidate #27988 fixes the issue (tested with 106X): produced packedCandidate has pt=65504 (maximal value allowed by packing) with correct values of eta and phi, the same for track retrieved from this packedCandidate.

=> I think it closes the issue.

[hatakeyamak]

Thanks. All sound good.

[hatakeyamak]

Perhaps this can be closed?

[mbluj]

Perhaps this can be closed?

Yes, right. I close it.