Nano branches from ValueMap of elaborated types #43916

vlimant · 2024-02-08T14:33:29Z

PR description:

in order to enable table extension, using elaborated types (other than int, float, ...) some specific templated producer need to be added. no change of existing branch/module are introduced.

PR validation:

the nano matrix succeeded.

cmsbuild · 2024-02-08T14:33:50Z

cms-bot internal usage

PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h

vlimant · 2024-02-08T14:34:54Z

PhysicsTools/NanoAOD/plugins/SimpleFlatTableProducerPlugins.cc

@@ -3,6 +3,9 @@
 #include "DataFormats/Candidate/interface/Candidate.h"
 typedef SimpleFlatTableProducer<reco::Candidate> SimpleCandidateFlatTableProducer;

+typedef SimpleTypedExternalFlatTableProducer<reco::Candidate, reco::Candidate>


this is a dummy producer that would have to be removed in the final PR

vlimant · 2024-02-08T14:35:30Z

meant to help with #43525

cmsbuild · 2024-02-08T14:41:48Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38766

ERROR: Build errors found during clang-tidy run.

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02823/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../include/c++/12.3.1/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'TypedValueMapVariable<reco::Candidate, reco::Candidate, StringObjectFunction<reco::Candidate>, int>::TypedValueMapVariable' requested here
--
PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h:125:9: error: no matching constructor for initialization of 'ValueMapVariable<Candidate, int, short>' [clang-diagnostic-error]
  125 |       : ValueMapVariable<ObjType, VarType, ValType>(aname, cfg, cc, skipNonExistingSrc),
      |         ^                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02823/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../include/c++/12.3.1/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'TypedValueMapVariable<reco::Candidate, reco::Candidate, StringObjectFunction<reco::Candidate>, int, short>::TypedValueMapVariable' requested here
--
PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h:125:9: error: no matching constructor for initialization of 'ValueMapVariable<Candidate, int, unsigned char>' [clang-diagnostic-error]
  125 |       : ValueMapVariable<ObjType, VarType, ValType>(aname, cfg, cc, skipNonExistingSrc),
      |         ^                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02823/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../include/c++/12.3.1/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'TypedValueMapVariable<reco::Candidate, reco::Candidate, StringObjectFunction<reco::Candidate>, int, unsigned char>::TypedValueMapVariable' requested here
--
PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h:125:9: error: no matching constructor for initialization of 'ValueMapVariable<Candidate, int, unsigned short>' [clang-diagnostic-error]
  125 |       : ValueMapVariable<ObjType, VarType, ValType>(aname, cfg, cc, skipNonExistingSrc),
      |         ^                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02823/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../include/c++/12.3.1/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'TypedValueMapVariable<reco::Candidate, reco::Candidate, StringObjectFunction<reco::Candidate>, int, unsigned short>::TypedValueMapVariable' requested here
--
PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h:125:9: error: no matching constructor for initialization of 'ValueMapVariable<Candidate, unsigned int, unsigned int>' [clang-diagnostic-error]
  125 |       : ValueMapVariable<ObjType, VarType, ValType>(aname, cfg, cc, skipNonExistingSrc),
      |         ^                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02823/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/lib/gcc/x86_64-redhat-linux-gnu/12.3.1/../../../../include/c++/12.3.1/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'TypedValueMapVariable<reco::Candidate, reco::Candidate, StringObjectFunction<reco::Candidate>, unsigned int>::TypedValueMapVariable' requested here
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:129: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

vlimant · 2024-02-08T14:57:17Z

FYI @mbluj

cmsbuild · 2024-02-08T15:05:09Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38767

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h modified in PR(s): Lepton time-life info for NanoAOD #43525
- File PhysicsTools/NanoAOD/plugins/SimpleFlatTableProducerPlugins.cc modified in PR(s): Lepton time-life info for NanoAOD #43525

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38767/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38767/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

vlimant · 2024-02-09T11:32:18Z

enable nano

cmsbuild · 2024-02-09T11:39:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38781

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h modified in PR(s): Lepton time-life info for NanoAOD #43525
- File PhysicsTools/NanoAOD/plugins/SimpleFlatTableProducerPlugins.cc modified in PR(s): Lepton time-life info for NanoAOD #43525

cmsbuild · 2024-02-09T11:39:46Z

A new Pull Request was created by @vlimant (vlimant) for master.

It involves the following packages:

PhysicsTools/NanoAOD (xpog)

@cmsbuild, @hqucms, @vlimant can you please review it and eventually sign? Thanks.
@AnnikaStein, @gpetruc this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

vlimant · 2024-02-09T11:40:00Z

please test

cmsbuild · 2024-02-12T13:36:02Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43916/38810

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h modified in PR(s): Lepton time-life info for NanoAOD #43525
- File PhysicsTools/NanoAOD/plugins/SimpleFlatTableProducerPlugins.cc modified in PR(s): Lepton time-life info for NanoAOD #43525

cmsbuild · 2024-02-12T13:36:12Z

Pull request #43916 was updated. @hqucms, @cmsbuild, @vlimant can you please check and sign again.

vlimant · 2024-02-12T13:36:38Z

please test

vlimant · 2024-02-12T13:38:39Z

@mbluj : I think that the best way is to merge this, and you rebase #43525 to pick up and use the feature (removing SimpleCandidate2CandidateFlatTableProducer) along the way

mbluj · 2024-02-12T14:03:53Z

@mbluj : I think that the best way is to merge this, and you rebase #43525 to pick up and use the feature (removing SimpleCandidate2CandidateFlatTableProducer) along the way

OK, so I am waiting until this is merged. Could it be also backported to 14_0?

cmsbuild · 2024-02-12T19:57:51Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e7e804/37371/summary.html
COMMIT: 20f3d13
CMSSW: CMSSW_14_1_X_2024-02-12-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43916/37371/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 92 lines from the logs
Reco comparison results: 37 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3248626
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248604
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 4 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 15
DQMHistoTests: Total histograms compared: 16524
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16524
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
Checked 38 log files, 18 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.0	2.545	2.545	0.000 ( +0.0% )	5.23	5.16	+1.3%	2.135	2.152
2500.001	2.692	2.692	0.000 ( +0.0% )	4.68	4.75	-1.6%	2.561	2.581
2500.002	2.631	2.631	0.000 ( +0.0% )	4.91	4.93	-0.4%	2.571	2.583
2500.01	1.315	1.315	0.000 ( +0.0% )	9.58	9.53	+0.6%	2.258	2.304
2500.011	1.732	1.732	0.000 ( +0.0% )	5.15	5.30	-2.9%	2.361	2.437
2500.012	1.578	1.578	0.000 ( +0.0% )	7.43	7.52	-1.1%	2.287	2.321
2500.1	2.193	2.193	0.000 ( +0.0% )	5.30	5.32	-0.3%	1.976	2.045
2500.2	2.307	2.307	0.000 ( +0.0% )	6.05	6.06	-0.3%	1.873	1.961
2500.21	1.183	1.183	0.000 ( +0.0% )	4.30	4.36	-1.5%	2.144	2.245
2500.211	1.547	1.547	0.000 ( +0.0% )	3.78	3.74	+1.1%	2.159	2.326
2500.3	2.063	2.063	0.000 ( +0.0% )	12.55	12.71	-1.3%	1.878	1.945
2500.31	1.258	1.258	0.000 ( +0.0% )	19.65	20.42	-3.8%	2.225	2.335
2500.311	1.646	1.646	0.000 ( +0.0% )	13.51	13.70	-1.4%	2.276	2.394
2500.312	7.159	7.159	0.000 ( +0.0% )	1.43	1.41	+0.8%	1.675	1.676
2500.313	1.564	1.564	0.000 ( +0.0% )	6.67	6.53	+2.1%	1.023	1.030
2500.4	2.063	2.063	0.000 ( +0.0% )	12.65	12.67	-0.2%	1.877	1.955
2500.5	19.575	19.575	0.000 ( +0.0% )	1.19	1.22	-2.7%	1.071	1.116

vlimant · 2024-02-13T10:33:04Z

+1

cmsbuild · 2024-02-13T10:33:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mbluj · 2024-02-13T10:36:32Z

Many thanks @vlimant! I am moving my development on top of it.
Could this development be also backported to 14_0 series, please?

antoniovilela · 2024-02-13T13:59:11Z

+1

mbluj · 2024-02-21T12:16:01Z

For reference: backported to 14_0_X in #44040.

add a way to add branches from valuemap to elaborated types

b9de1e0

cmsbuild added this to the CMSSW_14_1_X milestone Feb 8, 2024

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending xpog-pending labels Feb 8, 2024

vlimant commented Feb 8, 2024

View reviewed changes

PhysicsTools/NanoAOD/interface/SimpleFlatTableProducer.h Outdated Show resolved Hide resolved

vlimant commented Feb 8, 2024

View reviewed changes

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 8, 2024

pass rvalue around

fe9f7a6

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 8, 2024

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 8, 2024

code format

00c6978

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 9, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 9, 2024

selptrs used also if extvars_ is empty

20f3d13

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 12, 2024

cmsbuild added tests-started and removed tests-pending labels Feb 12, 2024

vlimant changed the title ~~Nano branches from ValueMap of elaborated types [WIP]~~ Nano branches from ValueMap of elaborated types Feb 12, 2024

cmsbuild added tests-approved and removed tests-started labels Feb 12, 2024

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Feb 13, 2024

mbluj mentioned this pull request Feb 13, 2024

Lepton time-life info for NanoAOD #43525

Merged

cmsbuild added orp-approved and removed orp-pending labels Feb 13, 2024

cmsbuild merged commit c1dbdf4 into cms-sw:master Feb 13, 2024
13 checks passed

cmsbuild mentioned this pull request Feb 14, 2024

Phase-2 TauMinator Tau (NNCaloTau) #42840

Merged

mbluj mentioned this pull request Feb 21, 2024

Lepton time-life info for NanoAOD and Nano branches from ValueMap of elaborated types (14_0_X) #44040

Merged

vlimant deleted the typed_valuemap_producer branch March 4, 2024 14:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nano branches from ValueMap of elaborated types #43916

Nano branches from ValueMap of elaborated types #43916

vlimant commented Feb 8, 2024 •

edited

Loading

cmsbuild commented Feb 8, 2024 •

edited

Loading

vlimant Feb 8, 2024

vlimant commented Feb 8, 2024

cmsbuild commented Feb 8, 2024

vlimant commented Feb 8, 2024

cmsbuild commented Feb 8, 2024

vlimant commented Feb 9, 2024

cmsbuild commented Feb 9, 2024

cmsbuild commented Feb 9, 2024

vlimant commented Feb 9, 2024

cmsbuild commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

vlimant commented Feb 12, 2024

vlimant commented Feb 12, 2024

mbluj commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

vlimant commented Feb 13, 2024

cmsbuild commented Feb 13, 2024

mbluj commented Feb 13, 2024

antoniovilela commented Feb 13, 2024

mbluj commented Feb 21, 2024

Nano branches from ValueMap of elaborated types #43916

Nano branches from ValueMap of elaborated types #43916

Conversation

vlimant commented Feb 8, 2024 • edited Loading

PR description:

PR validation:

cmsbuild commented Feb 8, 2024 • edited Loading

vlimant Feb 8, 2024

Choose a reason for hiding this comment

vlimant commented Feb 8, 2024

cmsbuild commented Feb 8, 2024

vlimant commented Feb 8, 2024

cmsbuild commented Feb 8, 2024

vlimant commented Feb 9, 2024

cmsbuild commented Feb 9, 2024

cmsbuild commented Feb 9, 2024

vlimant commented Feb 9, 2024

cmsbuild commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

vlimant commented Feb 12, 2024

vlimant commented Feb 12, 2024

mbluj commented Feb 12, 2024

cmsbuild commented Feb 12, 2024

Comparison Summary

NANO Comparison Summary

vlimant commented Feb 13, 2024

cmsbuild commented Feb 13, 2024

mbluj commented Feb 13, 2024

antoniovilela commented Feb 13, 2024

mbluj commented Feb 21, 2024

vlimant commented Feb 8, 2024 •

edited

Loading

cmsbuild commented Feb 8, 2024 •

edited

Loading