update Run2 data and mc GTs with JERC tags

[malbouis]

PR description:

This PR is to update the Run2 data and MC GTs with the most up to date JERC tags, as requested in https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4401.html

These tags were first updated in 106X (PR #33710).

Comparison between the previous GTs (in 113X) and the new ones are below:

2016 realistic pre-VFP era
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/113X_mcRun2_asymptotic_preVFP_v4/120X_mcRun2_asymptotic_preVFP_v1

2016 realistic post-VFP era
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/113X_mcRun2_asymptotic_v4/120X_mcRun2_asymptotic_v1

Offline data
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/113X_dataRun2_v6/120X_dataRun2_v1

Offline data relval
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/113X_dataRun2_relval_v6/120X_dataRun2_relval_v1

PR validation:

runTheMatrix.py -l limited,136.772,136.7721,1325.516,135.1 --ibeos

if this PR is a backport please specify the original PR and why you need to backport that PR:

It is not a backport.

[malbouis]

test parameters:

• 136.772,136.7721,1325.516,135.1

[malbouis]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33982/23086

• This PR adds an extra 12KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/AlCa/python/autoCond.py modified in PR(s): [DD4hep] start on geometry XML payload producer #33548

[cmsbuild]

A new Pull Request was created by @malbouis for master.

It involves the following packages:

Configuration/AlCa

@yuanchao, @malbouis, @pohsun, @tlampen, @francescobrivio can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8d781/15654/summary.html
COMMIT: d82024a
CMSSW: CMSSW_12_0_X_2021-06-04-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33982/15654/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 1541 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2650486
• DQMHistoTests: Total failures: 3546
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2646917
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.004 KiB( 36 files compared)
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• Checked 155 log files, 37 edm output root files, 37 DQM output files
• TriggerResults: no differences found

[malbouis]

test parameters:

• workflow=136.772,136.7721,1325.516,135.1

[malbouis]

please test

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8d781/15663/summary.html
COMMIT: d82024a
CMSSW: CMSSW_12_0_X_2021-06-04-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33982/15663/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-gem_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-pixellumi_dqm_sourceclient had ERRORS

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a8d781/135.1_TTbar_13+TTbarFS_13+HARVESTUP15FS+MINIAODMCUP15FS

Summary:

• No significant changes to the logs found
• Reco comparison results: 1533 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2650486
• DQMHistoTests: Total failures: 3541
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2646923
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
• Checked 155 log files, 37 edm output root files, 37 DQM output files
• TriggerResults: no differences found

[malbouis]

I ran the unit tests locally and did not see these errors. Will retrigger the tests for the case these are transient errors.

[malbouis]

test parameters:

workflow=136.772,136.7721,1325.516,135.1

[malbouis]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a8d781/15725/summary.html
COMMIT: d82024a
CMSSW: CMSSW_12_0_X_2021-06-06-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33982/15725/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a8d781/135.1_TTbar_13+TTbarFS_13+HARVESTUP15FS+MINIAODMCUP15FS

Summary:

• No significant changes to the logs found
• Reco comparison results: 1537 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2648335
• DQMHistoTests: Total failures: 3541
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2644772
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
• Checked 155 log files, 37 edm output root files, 37 DQM output files
• TriggerResults: no differences found

[yuanchao]

+1

• update GT and JERC tags for Run2 data/mc
• matrix tests and unit tests passed

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[francescobrivio]

hold

[cmsbuild]

Pull request has been put on hold by @francescobrivio
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[francescobrivio]

unhold

• differences in 2017 MC WF are only related to the messageLogger

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1