Enable boosted tau reco to run at miniAOD #34576

mbluj · 2021-07-21T16:24:07Z

PR description:

This development enables boosted tau pair reco to be run on top of miniAOD. To achieve this goal the following changes were introduced:

BoostedTauSeedProducer was templated to enable miniAOD inputs,
Tool to adapt tau sequences was modified (functions grouped into class, possibility to use postfix added, boosted tau pair reco enabled, some cleaning).

These changes do not modifies standard workflows.

Note: boosted tau pair reco with miniAOD inputs compared to its counterpart with AOD inputs shows similar differences as observed for standard tau reco.

PR validation:

A MiniAOD production workflow and a custom tau-reco-at-mini one successfully tested.

cmsbuild · 2021-07-21T16:32:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24125

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoTauTag/Configuration/python/tools/adaptToRunAtMiniAOD.py modified in PR(s): Remove six python2->3 compatibility helper usage #34534

cmsbuild · 2021-07-21T16:33:04Z

A new Pull Request was created by @mbluj for master.

It involves the following packages:

RecoJets/JetProducers (reconstruction)
RecoTauTag/Configuration (reconstruction)
RecoTauTag/RecoTau (reconstruction)

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @jdolen, @yslai, @swozniewski, @jdamgov, @ahinzmann, @nhanvtran, @gkasieczka, @clelange, @schoef, @mbluj, @mariadalfonso, @seemasharmafnal this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2021-07-22T12:05:35Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24138

This PR adds an extra 28KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24138/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24138/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly

mbluj · 2021-07-22T12:44:37Z

Development branch was rebased to master and conflicts solved.

cmsbuild · 2021-07-22T12:46:00Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24139

This PR adds an extra 28KB to repository

cmsbuild · 2021-07-22T12:46:25Z

Pull request #34576 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2021-07-22T17:03:03Z

@cmsbuild please test

cmsbuild · 2021-07-22T20:52:33Z

-1

Failed Tests: UnitTests RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65d19d/17101/summary.html
COMMIT: 1d7f95a
CMSSW: CMSSW_12_0_X_2021-07-22-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34576/17101/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@davidlange6 Remove six python2->3 compatibility helper usage #34534

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65d19d/17101/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65d19d/17101/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_MuonGeometryDBConverter had ERRORS
---> test testDD4hepCompactView had ERRORS
---> test testDD4hepFilteredView had ERRORS
---> test testDD4hepFilteredViewFind had ERRORS
and more ...

RelVals

11634.911

----- Begin Fatal Exception 22-Jul-2021 21:54:53 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34576/17101/CMSSW_12_0_X_2021-07-22-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34576/17101/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34576/17101/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34576/17101/CMSSW_12_0_X_2021-07-22-1100/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

slava77 · 2021-07-22T22:05:49Z

@cmsbuild please test

mbluj · 2021-08-10T11:09:44Z

Rebased, conflicts should be fixed by it

cmsbuild · 2021-08-10T11:14:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34576/24601

This PR adds an extra 32KB to repository
Found files with invalid states:
- RecoTauTag/Configuration/python/tools/helpers.py:
  - Added: 20c6812
  - Modified: d346436
  - Deleted: 890ba1d

cmsbuild · 2021-08-10T11:14:54Z

Pull request #34576 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2021-08-10T12:48:02Z

@cmsbuild please test

slava77 · 2021-08-11T12:42:05Z

@cmsbuild please test

24 hrs seems too long for the last request

cmsbuild · 2021-08-11T22:28:34Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-65d19d/17707/summary.html
COMMIT: 717ee73
CMSSW: CMSSW_12_1_X_2021-08-10-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34576/17707/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

11834.2411834.24_TTbar_14TeV+2021PU_0T+TTbar_14TeV_TuneCP5_GenSimINPUT+DigiPU+RecoPU+HARVESTPU+Nano/step2_TTbar_14TeV+2021PU_0T+TTbar_14TeV_TuneCP5_GenSimINPUT+DigiPU+RecoPU+HARVESTPU+Nano.log
23434.992123434.9921_TTbar_14TeV+2026D49PU_PMXS1S2ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+MiniAODPU/step2_TTbar_14TeV+2026D49PU_PMXS1S2ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+MiniAODPU.log
23434.2123434.21_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU/step2_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU.log

Expand to see more relval errors ...

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2999420
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2999397
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

slava77 · 2021-08-11T23:23:37Z

+reconstruction

for #34576 717ee73

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show no differences
- the updated parts of tau reco from mini are still experimental, they are not in the matrix workflows with comparisons
- a unit test runtestRecoTauTagRecoTau includes rerunTauRecoOnMiniAOD.py and it passed (although it's not obvious if the boosted taus actually ran)

cmsbuild · 2021-08-11T23:23:59Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-08-12T09:06:10Z

+1

perrotta · 2021-08-12T09:07:55Z

merge

cmsbuild added this to the CMSSW_12_0_X milestone Jul 21, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 21, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 21, 2021

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 22, 2021

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 22, 2021

mbluj force-pushed the CMSSW_12_0_X_tau-pog_boostedTausAtMini branch from 0f0d032 to 1d7f95a Compare July 22, 2021 12:38

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 22, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2021

cmsbuild added code-checks-pending and removed xpog-pending heterogeneous-pending labels Aug 10, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 10, 2021

cmsbuild added tests-started and removed tests-pending labels Aug 10, 2021

cmsbuild added tests-rejected and removed tests-started labels Aug 11, 2021

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Aug 11, 2021

cmsbuild added orp-approved and removed orp-pending labels Aug 12, 2021

cmsbuild merged commit 0bfad9d into cms-sw:master Aug 12, 2021

This was referenced Aug 12, 2021

Add GEM unpacker in L1Repack FullMC and temporary load of GEM geometry to hltGetConfiguration #34788

Closed

[GCC11] Fix warning: loop variable binds to a temporary #34852

Merged

initialize Trackster::iterationIndex_ #34858

Merged

mbluj deleted the CMSSW_12_0_X_tau-pog_boostedTausAtMini branch October 10, 2023 10:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable boosted tau reco to run at miniAOD #34576

Enable boosted tau reco to run at miniAOD #34576

mbluj commented Jul 21, 2021

cmsbuild commented Jul 21, 2021

cmsbuild commented Jul 21, 2021

cmsbuild commented Jul 22, 2021

mbluj commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

slava77 commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

slava77 commented Jul 22, 2021

mbluj commented Aug 10, 2021

cmsbuild commented Aug 10, 2021

cmsbuild commented Aug 10, 2021

slava77 commented Aug 10, 2021

slava77 commented Aug 11, 2021

cmsbuild commented Aug 11, 2021

slava77 commented Aug 11, 2021

cmsbuild commented Aug 11, 2021

perrotta commented Aug 12, 2021

perrotta commented Aug 12, 2021

Enable boosted tau reco to run at miniAOD #34576

Enable boosted tau reco to run at miniAOD #34576

Conversation

mbluj commented Jul 21, 2021

PR description:

PR validation:

cmsbuild commented Jul 21, 2021

cmsbuild commented Jul 21, 2021

cmsbuild commented Jul 22, 2021

mbluj commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

slava77 commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Unit Tests

RelVals

slava77 commented Jul 22, 2021

mbluj commented Aug 10, 2021

cmsbuild commented Aug 10, 2021

cmsbuild commented Aug 10, 2021

slava77 commented Aug 10, 2021

slava77 commented Aug 11, 2021

cmsbuild commented Aug 11, 2021

RelVals-INPUT

Comparison Summary

slava77 commented Aug 11, 2021

cmsbuild commented Aug 11, 2021

perrotta commented Aug 12, 2021

perrotta commented Aug 12, 2021