Add DY option to Embedding HepMC filter #38829

IzaakWN · 2022-07-22T07:27:36Z

PR description

This PR adds the IncludeDY option to the EmbeddingHepMCFilter in order to allow for the inclusion of Drell-Yan MC events with virtual photons / Z bosons for which the intermediate boson is missing in the particle collection, e.g. p p > ta ta~). For example, this is needed for correctly filtering events of the DYJetsToTauTauToMuTauh_M-50 sample. The proposed changes were discussed in the TauPOG here: https://indico.cern.ch/event/1170879/#2-validation-of-exclusive-dy-t

We have also added the absolute value function around the mother's PDG ID to allow for charged bosons, such as W (±24) for H → WW.

PR validation

We have compared MC events generated with and without the changes to this filter.
Usual scram build code-checks and scram build code-format.

PR backporting

After PR, we will request to backport these changes to CMSSW_10_6_X as well, so we can regenerate the DYJetsToTauTauToMuTauh_M-50 sample for Run-2 UL MC for TauPOG studies, and also to the CMSSW version for the future Run-3 MC campaign.

…e scope

cmsbuild · 2022-07-22T07:34:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38829/31201

This PR adds an extra 16KB to repository

cmsbuild · 2022-07-22T07:34:43Z

A new Pull Request was created by @IzaakWN (Izaak) for master.

It involves the following packages:

GeneratorInterface/Core (generators)

@SiewYan, @mkirsano, @Saptaparna, @cmsbuild, @GurpreetSinghChahal, @alberto-sanchez can you please review it and eventually sign? Thanks.
@alberto-sanchez, @mkirsano this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

SiewYan · 2022-07-25T01:36:49Z

please test

SiewYan · 2022-07-25T01:52:06Z

@IzaakWN , do you happen to have a test snippet included with this PR to test the switch to this filter?

cmsbuild · 2022-07-25T05:47:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ff0c5/26427/summary.html
COMMIT: 24cbe44
CMSSW: CMSSW_12_5_X_2022-07-24-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38829/26427/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3667670
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3667640
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 210 log files, 47 edm output root files, 51 DQM output files
TriggerResults: no differences found

IzaakWN · 2022-07-25T17:55:11Z

@IzaakWN , do you happen to have a test snippet included with this PR to test the switch to this filter?

Sure, I grabbed the fragment for DYJetsToTauTauToMuTauh_M-50 from MCM

curl -s -k https://cms-pdmv.cern.ch/mcm/public/restapi/requests/get_test/TAU-RunIISummer20UL18wmLHEGEN-00006 > setup_DYJetsToMuTauh.sh
bash setup_DYJetsToMuTauh_buggy.sh

and simply added IncludeDY = cms.bool(True), as follows:

process.generator = cms.EDFilter("Pythia8HadronizerFilter",
    HepMCFilter = cms.PSet(
        filterName = cms.string('EmbeddingHepMCFilter'),
        filterParameters = cms.PSet(
            BosonPDGID = cms.int32(23),
            IncludeDY = cms.bool(True),
            ElElCut = cms.string(''),
            ElHadCut = cms.string(''),
            ElMuCut = cms.string(''),
            Final_States = cms.vstring('MuHad'),
            HadHadCut = cms.string(''),
            MuHadCut = cms.string('Mu.Pt > 16 && Had.Pt > 16 && Mu.Eta < 2.5 && Had.Eta < 2.7'),
            MuMuCut = cms.string('')
        )
    ),
    # ...
)

and then run with something like

cmsRun -n3 -e -j test TAU-RunIISummer20UL18wmLHEGEN-00006_1_cfg.py

You can find more instruction in the back up of this presentation:
https://indico.cern.ch/event/1170879/#2-validation-of-exclusive-dy-t

SiewYan · 2022-07-26T06:34:22Z

Thanks you for the slide, it looks fine for me.

qliphy · 2022-08-01T01:19:01Z

please test
to refresh

qliphy · 2022-08-01T01:19:14Z

@cms-sw/generators-l2 Do you have any comment?

cmsbuild · 2022-08-01T05:25:17Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ff0c5/26553/summary.html
COMMIT: 24cbe44
CMSSW: CMSSW_12_5_X_2022-07-31-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38829/26553/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3669004
DQMHistoTests: Total failures: 25
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3668956
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 50 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 210 log files, 47 edm output root files, 51 DQM output files
TriggerResults: no differences found

IzaakWN · 2022-08-02T13:00:54Z

Just for the record, I have already opened these two backports:

CMMS_10_6_X for Run 2 UL: cms-sw/cmssw#38936
CMMS_12_4_X for Run 3: cms-sw/cmssw#38938

Saptaparna · 2022-08-02T15:24:04Z

GeneratorInterface/Core/src/EmbeddingHepMCFilter.cc

+      }
+      if (mom_id == ZPDGID_) {
+        isHardProc = true;  // intermediate boson
+      } else if (includeDY_ && 11 <= pdg_id && pdg_id <= 16 && mcTruthHelper_.isFirstCopy(**particle) &&


11 <= pdg_id && pdg_id <= 16: instead of doing this, do you want to select the pdg_ids explicitly. Do you care about Z $\rightarrow \nu \nu$?

In principle this filter only targets the charged leptons, but this is done a few lines below and selecting 11, 13 and 15 explicitly should make no significant difference in performance.

I can change it if you really want?

@Saptaparna, are you fine with leaving it as it is now?

Yes, sorry. It is fine. Just signed PR.

qliphy · 2022-08-15T10:39:28Z

please test

qliphy · 2022-08-15T10:40:03Z

@cms-sw/generators-l2 Any comment?
#38829 (comment)

cmsbuild · 2022-08-15T15:14:25Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ff0c5/26821/summary.html
COMMIT: 24cbe44
CMSSW: CMSSW_12_5_X_2022-08-15-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38829/26821/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3692476
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3692452
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 49 edm output root files, 51 DQM output files
TriggerResults: no differences found

Saptaparna · 2022-08-16T15:16:36Z

+1
but some tests failed in the 12X back port.

cmsbuild · 2022-08-16T15:45:54Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-08-16T15:46:07Z

+1

qliphy · 2022-08-17T01:06:56Z

@IzaakWN There appears an IB issue after merging this PR:
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_5_X_2022-08-16-2300/pyRelValMatrixLogs/run/136.9_RunDoubleMuon2016C+RunDoubleMuon2016C+RAWRECOTE16+RAWRECOLHECLEANTE16+EMBEDHADTE16+EMBEDMINIAOD16/step4_RunDoubleMuon2016C+RunDoubleMuon2016C+RAWRECOTE16+RAWRECOLHECLEANTE16+EMBEDHADTE16+EMBEDMINIAOD16.log#/

Please have a quick fix. Thanks!

aandvalenzuela · 2022-08-17T07:15:00Z

Hi @IzaakWN,
Also IB unit test is failing: https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_5_X_2022-08-16-2300/unitTestLogs/TauAnalysis/MCEmbeddingTools#/171-171 because of the same reason.

IzaakWN · 2022-08-17T07:35:28Z

@qliphy @aandvalenzuela, I guess it is because the IncludeDY parameter is not specified in these test scripts:

cmssw/TauAnalysis/MCEmbeddingTools/python/EmbeddingPythia8Hadronizer_cfi.py

Lines 8 to 24 in 6d2f660

    
           generator = cms.EDFilter("Pythia8HadronizerFilter", 
        
             maxEventsToPrint = cms.untracked.int32(1), 
        
             nAttempts = cms.uint32(1000), 
        
             HepMCFilter = cms.PSet( 
        
               filterName = cms.string('EmbeddingHepMCFilter'), 
        
               filterParameters = cms.PSet( 
        
                       ElElCut = cms.string('El1.Pt > 22 && El2.Pt > 10'), 
        
                       ElHadCut = cms.string('El.Pt > 28 && Had.Pt > 25'), 
        
                       ElMuCut = cms.string('(El.Pt > 21 && Mu.Pt > 10) || (El.Pt > 10 && Mu.Pt > 21)'), 
        
                       HadHadCut = cms.string('Had1.Pt > 35 && Had2.Pt > 30'), 
        
                       MuHadCut = cms.string('Mu.Pt > 18 && Had.Pt > 25 && Mu.Eta < 2.1'), 
        
                       MuMuCut = cms.string('Mu1.Pt > 17 && Mu2.Pt > 8'), 
        
                       Final_States = cms.vstring('ElEl','ElHad','ElMu','HadHad','MuHad','MuMu'), 
        
                       BosonPDGID = cms.int32(23) 
        
               ), 
        
             ),

Is it possible to make it optional here?

cmssw/GeneratorInterface/Core/src/EmbeddingHepMCFilter.cc

Line 7 in af34412

    
           : ZPDGID_(iConfig.getParameter<int>("BosonPDGID")), includeDY_(iConfig.getParameter<bool>("IncludeDY")) {

perrotta · 2022-08-17T07:57:51Z

@IzaakWN the optimal solution would be to add a fillDescriptions method in your Pythia8HadronizerFilter.

A quicker fix would be to simply add the missing parameter in cmssw/TauAnalysis/MCEmbeddingTools/python/EmbeddingPythia8Hadronizer_cfi.py, assigning it a "false" default value. Could you please take care of it?

IzaakWN · 2022-08-17T08:04:16Z

Hi @perrotta, I prefer to solve it by making the IncludeDY parameter optional like this, so it does not conflict with previous fragment configurations:

iConfig.getUntrackedParameter<bool>("IncludeDY", false)

1828dd5
Is this okay with you? Then I'll open a new PR, and update the backports as well.

perrotta · 2022-08-17T08:18:07Z

@IzaakWN in your fix please both update EmbeddingPythia8Hadronizer_cfi.py and assign a default value for "IncludeDY" as you are suggesting

IzaakWN · 2022-08-17T09:31:46Z

Hi @qliphy, @aandvalenzuela, @perrotta, the fix is in PR #39090 and it has been pushed to the backports as well. Apologies for the troubles.

IzaakWN added 3 commits July 20, 2022 12:25

add DY option for virtual photons / off-shell events

7b91923

add absolute value to include negative W boson; move if clause outsid…

b14bb5f

…e scope

replace cout with LogInfo; improve comments

24cbe44

cmsbuild added this to the CMSSW_12_5_X milestone Jul 22, 2022

cmsbuild added code-checks-pending generators-pending orp-pending pending-signatures tests-pending labels Jul 22, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 25, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 25, 2022

cmsbuild added tests-started and removed tests-approved labels Aug 1, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 1, 2022

IzaakWN mentioned this pull request Aug 2, 2022

[106X backport] Add DY option to Embedding HepMC filter #38936

Merged

Saptaparna reviewed Aug 2, 2022

View reviewed changes

cmsbuild added tests-started and removed tests-approved labels Aug 15, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 15, 2022

cmsbuild added fully-signed generators-approved and removed generators-pending pending-signatures labels Aug 16, 2022

cmsbuild added orp-approved and removed orp-pending labels Aug 16, 2022

cmsbuild merged commit 8ab7197 into cms-sw:master Aug 16, 2022

cmsbuild mentioned this pull request Aug 16, 2022

Remove run2_HLTconditions_2018 modifier from Run3 setting #39084

Merged

IzaakWN mentioned this pull request Aug 17, 2022

[Fix] Make IncludeDY optional #39090

Merged

This was referenced Aug 17, 2022

Update cuda to 11.7.1 for gcc12 cms-sw/cmsdist#8025

Closed

[GCC Flags] drop -Wstrict-overflow which has been deprecated since gcc8 cms-sw/cmsdist#8026

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DY option to Embedding HepMC filter #38829

Add DY option to Embedding HepMC filter #38829

IzaakWN commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

SiewYan commented Jul 25, 2022

SiewYan commented Jul 25, 2022

cmsbuild commented Jul 25, 2022

IzaakWN commented Jul 25, 2022

SiewYan commented Jul 26, 2022

qliphy commented Aug 1, 2022

qliphy commented Aug 1, 2022

cmsbuild commented Aug 1, 2022

IzaakWN commented Aug 2, 2022

Saptaparna Aug 2, 2022

IzaakWN Aug 2, 2022

IzaakWN Aug 12, 2022

Saptaparna Aug 16, 2022

qliphy commented Aug 15, 2022

qliphy commented Aug 15, 2022

cmsbuild commented Aug 15, 2022

Saptaparna commented Aug 16, 2022 •

edited by qliphy

Loading

cmsbuild commented Aug 16, 2022

qliphy commented Aug 16, 2022

qliphy commented Aug 17, 2022

aandvalenzuela commented Aug 17, 2022

IzaakWN commented Aug 17, 2022

perrotta commented Aug 17, 2022

IzaakWN commented Aug 17, 2022 •

edited

Loading

perrotta commented Aug 17, 2022

IzaakWN commented Aug 17, 2022

Add DY option to Embedding HepMC filter #38829

Add DY option to Embedding HepMC filter #38829

Conversation

IzaakWN commented Jul 22, 2022

PR description

PR validation

PR backporting

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

SiewYan commented Jul 25, 2022

SiewYan commented Jul 25, 2022

cmsbuild commented Jul 25, 2022

Comparison Summary

IzaakWN commented Jul 25, 2022

SiewYan commented Jul 26, 2022

qliphy commented Aug 1, 2022

qliphy commented Aug 1, 2022

cmsbuild commented Aug 1, 2022

Comparison Summary

IzaakWN commented Aug 2, 2022

Saptaparna Aug 2, 2022

Choose a reason for hiding this comment

IzaakWN Aug 2, 2022

Choose a reason for hiding this comment

IzaakWN Aug 12, 2022

Choose a reason for hiding this comment

Saptaparna Aug 16, 2022

Choose a reason for hiding this comment

qliphy commented Aug 15, 2022

qliphy commented Aug 15, 2022

cmsbuild commented Aug 15, 2022

Comparison Summary

Saptaparna commented Aug 16, 2022 • edited by qliphy Loading

cmsbuild commented Aug 16, 2022

qliphy commented Aug 16, 2022

qliphy commented Aug 17, 2022

aandvalenzuela commented Aug 17, 2022

IzaakWN commented Aug 17, 2022

perrotta commented Aug 17, 2022

IzaakWN commented Aug 17, 2022 • edited Loading

perrotta commented Aug 17, 2022

IzaakWN commented Aug 17, 2022

Saptaparna commented Aug 16, 2022 •

edited by qliphy

Loading

IzaakWN commented Aug 17, 2022 •

edited

Loading