Minimal era for Run 3 PbPb and corresponding relVal workflows #29032

mandrenguyen · 2020-02-25T08:12:39Z

PR description:

Adds Run3_PbPb to run PbPb reconstruction with 2021 conditions. This era contains the minimal modifications to produce a running workflow. It uses the same modifiers as in the Run2_2018_pp_on_AA era. Also adding 2021 heavy-ion MC relval workflows by copying the corresponding 2018 ones.

PR validation:

Tested locally with the new relval workflow 159.1, after adding PR #29078 by hand.

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild · 2020-02-25T08:13:06Z

The code-checks are being triggered in jenkins.

mandrenguyen · 2020-02-25T08:13:22Z

@abaty

cmsbuild · 2020-02-25T08:18:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29032/13896

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Eras.py modified in PR(s): Mydev #29030

cmsbuild · 2020-02-25T08:18:40Z

A new Pull Request was created by @mandrenguyen (Matthew Nguyen) for master.

It involves the following packages:

Configuration/Eras
Configuration/StandardSequences
RecoTracker/IterativeTracking

@perrotta, @silviodonato, @cmsbuild, @franzoni, @slava77, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @mschrode, @lecriste, @gpetruc, @ebrondol, @dgulhan this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

boudoul · 2020-02-25T08:51:20Z

Great , thanks @mandrenguyen ! Presumably we should define a workflow using it, do you want me to take care ?

mandrenguyen · 2020-02-25T08:54:12Z

@boudoul I can setup a relval workflow. Let's get some feedback first on this proposal for an era from @slava77 @perrotta

boudoul · 2020-02-25T08:55:57Z

Perfect, thank you

slava77

except it removes the HI-specific track MVA calibrations, which do not yet exist in a 2021 global tag

why do they not exist for the 2021 GT?
IIRC, the regular tracking selectors are the same in 2021 as in 2018.
Did I miss some essential tracking changes that would invalidate HIMVASelector[s]?

Configuration/Eras/python/Era_Run3_PbPb_cff.py

mandrenguyen · 2020-02-25T14:54:17Z

except it removes the HI-specific track MVA calibrations, which do not yet exist in a 2021 global tag

why do they not exist for the 2021 GT?
IIRC, the regular tracking selectors are the same in 2021 as in 2018.
Did I miss some essential tracking changes that would invalidate HIMVASelector[s]?

In my tests I have used the standard pp GT (and corresponding beamspot) in my tests. A crash in reco indicated to me that they do not appear to contain this record. I assume there is not matching GT/beamspot set yet for heavy-ion beam conditions. I don't believe there's anything fundamental. One could simply add these records to the pp GTs as well. Of course the selection has to be retrained for Run 3 in any case.

slava77 · 2020-02-25T16:22:59Z

One could simply add these records to the pp GTs as well.

I prefer this solution; it seems more appropriate for the current state of the physics performance setup.

mandrenguyen · 2020-02-25T20:02:32Z

@abaty Can you upload the HIMVA tags into the GT queue for 2021 realistic conditions, please?

abaty · 2020-02-26T16:39:15Z

@mandrenguyen I've asked Xiao Wang to upload the relevant tags to the GT queue.

tocheng · 2020-02-27T20:15:05Z

I prefer to have a separate GT for heavy Ion MC. AlCa can make a heavy Ion queue for 2021 based on pp and heavy Ion group can queue tags to add new conditions or replace conditions in pp GT.

mandrenguyen · 2020-02-27T20:21:58Z

@tocheng Is there a reason not to add this record to the pp GT? We may want to test the HI era with various pp GTs in the future. In any case, this is fine for us if @slava77 agrees. What would be the ETA to produce a HI GT that is the same as the one pointed to by auto:phase1_2021_realistic, except that it has the add'l HIMVA records?

tocheng · 2020-02-27T20:26:16Z

@mandrenguyen What do you mean various pp GTs?
We could make a new queue based on pp 2021 and you can add the missing tracker selector. The HI GT can be made soon after.

mandrenguyen · 2020-02-27T20:40:46Z

@tocheng I just meant that adding this record to pp GTs would allow us to run PbPb_run3 era with pp GTs without crashing. That sounds like something that could come in handy to me. I think we can drop this discussion though. We will need the HI queue anyway, since there will eventually be beamspot tag to upload there. Let's just get this era operational. Once a HI GT is available I can update the autocond, and add a relval wf.

tocheng · 2020-02-27T20:42:23Z

111X_mcRun3_2021_realistic_HI_Queue is ready and is equal to auto:phase1_2021
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/111X_mcRun3_2021_realistic_HI_Queue/110X_mcRun3_2021_realistic_v8

Taburis · 2020-02-27T20:58:07Z

Hi @tocheng, @mandrenguyen, and @slava77, the MVA tags added to the queue: 111X_mcRun3_2021_realistic_HI_Queue.

cmsbuild · 2020-03-10T22:28:35Z

+1
Tested at: 7370e65
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e9604/5116/summary.html
CMSSW: CMSSW_11_1_X_2020-03-10-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-10T22:28:42Z

Comparison job queued.

cmsbuild · 2020-03-11T00:04:12Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e9604/5116/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-3e9604/159.0_HydjetQ_B12_5020GeV_2021_ppReco+HydjetQ_B12_5020GeV_2021_ppReco+DIGIHI2021PPRECO+RECOHI2021PPRECO+ALCARECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-3e9604/159.1_QCD_Pt_80_120_14_HI_2021+QCD_Pt_80_120_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-3e9604/159.2_PhotonJets_Pt_10_14_HI_2021+PhotonJets_Pt_10_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-3e9604/159.3_ZMM_14_HI_2021+ZMM_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-3e9604/159.4_ZEE_14_HI_2021+ZEE_14_HI_2021+DIGIHI2021PPRECO+RECOHI2021PPRECO+HARVESTHI2021PPRECO

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2680577
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2680256
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

chayanit · 2020-03-11T08:08:01Z

+1

silviodonato · 2020-03-11T09:58:35Z

+operations

kpedro88 · 2020-03-11T14:25:20Z

+upgrade

silviodonato · 2020-03-12T14:05:25Z

Kind reminder for generators: @alberto-sanchez @agrohsje @efeyazgan @mkirsano @qliphy @SiewYan

qliphy · 2020-03-12T14:13:18Z

+1

cmsbuild · 2020-03-12T14:13:46Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-03-12T16:54:20Z

+1

mandrenguyen added 2 commits February 23, 2020 11:46

Running version of minimal Run 3 PbPb era

2496e72

Add PbPb run 3 customization to turn off HI-specific MVA track calibs

b9b0f4d

cmsbuild added this to the CMSSW_11_1_X milestone Feb 25, 2020

cmsbuild added code-checks-pending comparison-pending operations-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 25, 2020

slava77 reviewed Feb 25, 2020

View reviewed changes

Configuration/Eras/python/Era_Run3_PbPb_cff.py Outdated Show resolved Hide resolved

cmsbuild added the tests-started label Mar 10, 2020

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 11, 2020

cmsbuild added pdmv-approved and removed pdmv-pending labels Mar 11, 2020

cmsbuild added operations-approved and removed operations-pending labels Mar 11, 2020

cmsbuild added upgrade-approved and removed upgrade-pending labels Mar 11, 2020

cmsbuild added fully-signed generators-approved and removed generators-pending pending-signatures labels Mar 12, 2020

cmsbuild added orp-approved and removed orp-pending labels Mar 12, 2020

cmsbuild merged commit f6fc185 into cms-sw:master Mar 12, 2020

This was referenced Mar 16, 2020

Missing pileup dataset in wf 310, 311, 312 #29216

Closed

Disable temporarily workflow 310, 311, 312 #29218

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Minimal era for Run 3 PbPb and corresponding relVal workflows #29032

Minimal era for Run 3 PbPb and corresponding relVal workflows #29032

mandrenguyen commented Feb 25, 2020 •

edited

Loading

cmsbuild commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

boudoul commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

boudoul commented Feb 25, 2020

slava77 left a comment

mandrenguyen commented Feb 25, 2020

slava77 commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

abaty commented Feb 26, 2020

tocheng commented Feb 27, 2020

mandrenguyen commented Feb 27, 2020

tocheng commented Feb 27, 2020

mandrenguyen commented Feb 27, 2020 •

edited

Loading

tocheng commented Feb 27, 2020

Taburis commented Feb 27, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 11, 2020

chayanit commented Mar 11, 2020

silviodonato commented Mar 11, 2020

kpedro88 commented Mar 11, 2020

silviodonato commented Mar 12, 2020

qliphy commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

silviodonato commented Mar 12, 2020

Minimal era for Run 3 PbPb and corresponding relVal workflows #29032

Minimal era for Run 3 PbPb and corresponding relVal workflows #29032

Conversation

mandrenguyen commented Feb 25, 2020 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

boudoul commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

boudoul commented Feb 25, 2020

slava77 left a comment

Choose a reason for hiding this comment

mandrenguyen commented Feb 25, 2020

slava77 commented Feb 25, 2020

mandrenguyen commented Feb 25, 2020

abaty commented Feb 26, 2020

tocheng commented Feb 27, 2020

mandrenguyen commented Feb 27, 2020

tocheng commented Feb 27, 2020

mandrenguyen commented Feb 27, 2020 • edited Loading

tocheng commented Feb 27, 2020

Taburis commented Feb 27, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 10, 2020

cmsbuild commented Mar 11, 2020

chayanit commented Mar 11, 2020

silviodonato commented Mar 11, 2020

kpedro88 commented Mar 11, 2020

silviodonato commented Mar 12, 2020

qliphy commented Mar 12, 2020

cmsbuild commented Mar 12, 2020

silviodonato commented Mar 12, 2020

mandrenguyen commented Feb 25, 2020 •

edited

Loading

mandrenguyen commented Feb 27, 2020 •

edited

Loading