Add ParticleNet info to taus in NanoAOD [13_0_X] #41394

mbluj · 2023-04-24T15:44:06Z

PR description:

This PR adds ParticleNet info (extended in #40745) to taus stored in NanoAOD. The additional information are as follows:

ParticleNet discriminators vs jets, electrons and muons,
decay mode corresponding with the ParticleNet tau node with the highest score,
correction for tau pt,
charge assignment confidence (signed),
ParticleNet scores for individual decay-modes (normalised to the sum of the scores for all tau nodes).

In addition to adding the above information to already existing taus reconstructed with the HPS algorithm (and passing the loosest WP of at least one of its discriminants vs jets) "recovery" taus are added which correspond with CHSAK4 jets with ParticleNet discriminant vs jets passing WP (>0.05) with an efficiency of ~97% for genuine taus. Also HPS taus passing this WP of ParticleNet vs jet discriminant and not passing other discriminants vs jets are stored (however their fraction is at permille level).

The PanrticleNet information and "recovery" taus are added with a new "hybrid tau" producer which matches CHSAK4 jets with ParticleNet taggers with HPS-taus. The producer can be in principle used also as part of miniAOD sequences.

This is backport of #41333 to CMSSW_13_0 series.

PR validation:

Validated with custom nano-AOD workflows with 13_0_X, more detailed validation performed for original PR.

PR backport

This is backport of #41333 to CMSSW_13_0 series for production of nanoAOD v12.

cmsbuild · 2023-04-24T15:44:36Z

A new Pull Request was created by @mbluj for CMSSW_13_0_X.

It involves the following packages:

PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (xpog, reconstruction)

@swertz, @vlimant, @clacaputo, @cmsbuild, @simonepigazzini, @mandrenguyen can you please review it and eventually sign? Thanks.
@AlexDeMoor, @rappoccio, @gouskos, @jdolen, @JyothsnaKomaragiri, @ahinzmann, @AnnikaStein, @schoef, @emilbols, @jdamgov, @mbluj, @nhanvtran, @gkasieczka, @hatakeyamak, @gpetruc, @azotz, @mariadalfonso, @demuller, @andrzejnovak, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Add ParticleNet info to taus in NanoAOD #41333

mbluj · 2023-04-24T15:53:15Z

FYI, @simonepigazzini.
This is verbatim backport of #41333, but I guess that the new features should be enabled for specific eras for nanoAOD v12 and disabled for historical eras. Could you (i.e. XPOG experts) please guide me for which eras switch them on/off, please?

simonepigazzini · 2023-04-25T08:03:38Z

Hi @mbluj, v12 is the version produced with 13_0_X without modifiers, therefore for that we are good. Concerning Run2 processing, the only problem I see is that the training you have included is suboptimal, since is performed on Run3. You can switch off the PNet computation for run2_nanoAOD_ANY around these lines:

cmssw/PhysicsTools/NanoAOD/python/nano_cff.py

Line 235 in 388e6f5

)

mbluj · 2023-04-25T08:26:57Z

Hi @mbluj, v12 is the version produced with 13_0_X without modifiers, therefore for that we are good. Concerning Run2 processing, the only problem I see is that the training you have included is suboptimal, since is performed on Run3. You can switch off the PNet computation for run2_nanoAOD_ANY around these lines:

cmssw/PhysicsTools/NanoAOD/python/nano_cff.py

Line 235 in 388e6f5

)

Yes, I know that the PNet model used is trained for Run3 conditions and that it is (probably) sub-optimal for Run2 conditions, but there is not alternative as Run-2 training is not present. Therefore, it is fine to have it also for v12 production with Run2 samples (if it will happen). However, my question was different: should the PNet-related updates for taus be switched off for historical eras, e.g. run2_nanoAOD_106Xv2 or run3_nanoAOD_122, to preserve their backward compatibility or it is fine to keep the configuration as it is.

swertz · 2023-04-25T12:11:20Z

Therefore, it is fine to have it also for v12 production with Run2 samples (if it will happen).

I think that makes sense, in any case there won't be a central production campaign of Run2 in V12.

What we need to make sure is that it's still possible to produce NanoV12 on Run2 UL miniAOD inputs, in case people want to run privately. But since the Run2 UL test workflows ran fine in #41333 it looks like that's OK (no need for modifiers). This is what the backwards-compatibility is about.

swertz · 2023-04-25T13:03:08Z

enable nano

swertz · 2023-04-25T13:03:13Z

please test

mbluj · 2023-04-25T13:43:47Z

Therefore, it is fine to have it also for v12 production with Run2 samples (if it will happen).

I think that makes sense, in any case there won't be a central production campaign of Run2 in V12.

What we need to make sure is that it's still possible to produce NanoV12 on Run2 UL miniAOD inputs, in case people want to run privately. But since the Run2 UL test workflows ran fine in #41333 it looks like that's OK (no need for modifiers). This is what the backwards-compatibility is about.

OK, thanks.
Yes, I confirm it works fine on Run2 UL miniAOD samples with the run2_nanoAOD_106Xv2 era modifier. So, I leave it as is.

cmsbuild · 2023-04-25T16:00:24Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b4bb00/32128/summary.html
COMMIT: 388e6f5
CMSSW: CMSSW_13_0_X_2023-04-25-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41394/32128/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

250202.0250202.0_TTbar_13/step2_TTbar_13.log
250200.0250200.0_ZEE_13/step2_ZEE_13.log
250200.17250200.17_ZEE_13UP17/step2_ZEE_13UP17.log

Expand to see more relval errors ...

Comparison Summary

Summary:

You potentially removed 6 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 109 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3554298
DQMHistoTests: Total failures: 142
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3554134
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 21.997999999999998 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 2.760 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 1.339 KiB Physics/NanoAODDQM
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 48 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 400 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12072
DQMHistoTests: Total failures: 505
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 11567
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 23.254999999999995 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 2.760 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.339 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.387	2.343	0.044 ( +1.9% )	5.57	5.68	-1.9%	1.514	1.453
2500.311	2.487	2.440	0.047 ( +1.9% )	5.02	5.12	-2.0%	1.905	1.769
2500.312	2.437	2.391	0.046 ( +1.9% )	5.16	5.32	-2.9%	1.896	1.764
2500.33	1.206	1.172	0.035 ( +3.0% )	10.60	10.78	-1.7%	1.653	1.479
2500.331	1.550	1.508	0.042 ( +2.8% )	5.71	5.80	-1.6%	1.806	1.652
2500.332	1.444	1.409	0.034 ( +2.4% )	8.20	8.33	-1.6%	1.848	1.571
2500.401	2.184	2.137	0.047 ( +2.2% )	5.37	10.46	-48.7%	1.385	1.137
2500.501	1.768	1.710	0.059 ( +3.4% )	9.16	17.05	-46.3%	1.290	1.077
2500.511	1.162	1.122	0.040 ( +3.5% )	4.45	31.32	-85.8%	1.584	1.321
2500.5111	1.520	1.477	0.043 ( +2.9% )	3.92	20.43	-80.8%	1.630	1.367
2500.601	2.049	2.038	0.011 ( +0.5% )	12.88	12.62	+2.1%	1.181	1.078

simonepigazzini · 2023-04-25T16:21:11Z

xrd glitch

simonepigazzini · 2023-04-25T16:21:19Z

please test

simonepigazzini · 2023-04-25T16:22:41Z

MINI/NANO AOD appears to be in order, I restarted the tests, if they fail again on the same (unrelated to this PR) wf we will approve anyway

cmsbuild · 2023-04-25T19:03:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b4bb00/32137/summary.html
COMMIT: 388e6f5
CMSSW: CMSSW_13_0_X_2023-04-25-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41394/32137/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 14 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 109 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3554298
DQMHistoTests: Total failures: 142
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3554134
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 21.997999999999998 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 2.760 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 1.339 KiB Physics/NanoAODDQM
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 50 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 400 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12072
DQMHistoTests: Total failures: 505
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 11567
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 23.254999999999995 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 2.760 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.339 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.387	2.343	0.044 ( +1.9% )	5.56	5.68	-2.1%	1.517	1.453
2500.311	2.487	2.440	0.047 ( +1.9% )	5.01	5.12	-2.3%	1.914	1.769
2500.312	2.437	2.391	0.046 ( +1.9% )	5.17	5.32	-2.8%	1.896	1.764
2500.33	1.206	1.172	0.035 ( +3.0% )	10.54	10.78	-2.2%	1.657	1.479
2500.331	1.550	1.508	0.042 ( +2.8% )	5.68	5.80	-2.1%	1.819	1.652
2500.332	1.444	1.409	0.034 ( +2.4% )	8.16	8.33	-2.1%	1.847	1.571
2500.401	2.184	2.137	0.047 ( +2.2% )	5.28	10.46	-49.5%	1.377	1.137
2500.501	1.768	1.710	0.059 ( +3.4% )	9.11	17.05	-46.5%	1.292	1.077
2500.511	1.162	1.122	0.040 ( +3.5% )	4.48	31.32	-85.7%	1.585	1.321
2500.5111	1.520	1.477	0.043 ( +2.9% )	3.83	20.43	-81.3%	1.630	1.367
2500.601	2.049	2.038	0.011 ( +0.5% )	12.57	12.62	-0.4%	1.181	1.078

perrotta · 2023-04-26T08:44:43Z

backport of #41333

mandrenguyen · 2023-04-27T13:59:15Z

type tau

mandrenguyen · 2023-04-27T13:59:27Z

+1

rappoccio · 2023-04-28T19:14:24Z

@simonepigazzini can you sign this? We can then merge in time for the next 13_0_X.

simonepigazzini · 2023-05-02T07:36:52Z

+1

cmsbuild · 2023-05-02T07:37:13Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-05-02T07:53:50Z

+1

All updates are related to nanoAOD

mbluj added 10 commits April 13, 2023 15:58

initial version of tau producer from HPS and PNet

5d7c9fb

Add fillDescriptions

1f14b45

Make selection configurable

601bcf6

Add genJet matching

fe69bc8

Use uncorrected jets to build recovery taus

465138b

Small addons and cleaning

5985521

Simplified tau content and other small updates

5feae71

Add PNet info to taus

6872adc

Add tau PNEt variables to DQM

0d93144

Store charge assignment confidence and tau decay mode scores

388e6f5

cmsbuild added this to the CMSSW_13_0_X milestone Apr 24, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending xpog-pending labels Apr 24, 2023

cmsbuild added tests-started and removed tests-pending labels Apr 25, 2023

cmsbuild added tests-rejected and removed tests-started labels Apr 25, 2023

cmsbuild added tests-started and removed tests-rejected labels Apr 25, 2023

cmsbuild added tests-approved and removed tests-started labels Apr 25, 2023

cmsbuild added the backport label Apr 26, 2023

cmsbuild added reconstruction-approved tau backport-ok and removed reconstruction-pending backport labels Apr 27, 2023

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels May 2, 2023

cmsbuild added orp-approved and removed orp-pending labels May 2, 2023

cmsbuild merged commit 53454b3 into cms-sw:CMSSW_13_0_X May 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ParticleNet info to taus in NanoAOD [13_0_X] #41394

Add ParticleNet info to taus in NanoAOD [13_0_X] #41394

mbluj commented Apr 24, 2023

cmsbuild commented Apr 24, 2023 •

edited

Loading

mbluj commented Apr 24, 2023 •

edited

Loading

simonepigazzini commented Apr 25, 2023

mbluj commented Apr 25, 2023

swertz commented Apr 25, 2023 •

edited

Loading

swertz commented Apr 25, 2023

swertz commented Apr 25, 2023

mbluj commented Apr 25, 2023

cmsbuild commented Apr 25, 2023

simonepigazzini commented Apr 25, 2023

simonepigazzini commented Apr 25, 2023

simonepigazzini commented Apr 25, 2023

cmsbuild commented Apr 25, 2023

perrotta commented Apr 26, 2023

mandrenguyen commented Apr 27, 2023

mandrenguyen commented Apr 27, 2023

rappoccio commented Apr 28, 2023

simonepigazzini commented May 2, 2023

cmsbuild commented May 2, 2023

perrotta commented May 2, 2023

Add ParticleNet info to taus in NanoAOD [13_0_X] #41394

Add ParticleNet info to taus in NanoAOD [13_0_X] #41394

Conversation

mbluj commented Apr 24, 2023

PR description:

PR validation:

PR backport

cmsbuild commented Apr 24, 2023 • edited Loading

mbluj commented Apr 24, 2023 • edited Loading

simonepigazzini commented Apr 25, 2023

mbluj commented Apr 25, 2023

swertz commented Apr 25, 2023 • edited Loading

swertz commented Apr 25, 2023

swertz commented Apr 25, 2023

mbluj commented Apr 25, 2023

cmsbuild commented Apr 25, 2023

RelVals-INPUT

Comparison Summary

NANO Comparison Summary

simonepigazzini commented Apr 25, 2023

simonepigazzini commented Apr 25, 2023

simonepigazzini commented Apr 25, 2023

cmsbuild commented Apr 25, 2023

Comparison Summary

NANO Comparison Summary

perrotta commented Apr 26, 2023

mandrenguyen commented Apr 27, 2023

mandrenguyen commented Apr 27, 2023

rappoccio commented Apr 28, 2023

simonepigazzini commented May 2, 2023

cmsbuild commented May 2, 2023

perrotta commented May 2, 2023

cmsbuild commented Apr 24, 2023 •

edited

Loading

mbluj commented Apr 24, 2023 •

edited

Loading

swertz commented Apr 25, 2023 •

edited

Loading