Use ECAL ratio timing algorithm for Run 1 and Run 2, and CC timing algorithm for Run 3 and beyond - 131X

[thomreis]

PR description:

This PR sets the default ECAL timing algorithm to the ratio method. This is the default timing algorithm for Run 1 and Run 2.
For Run 3 and Phase 2 a new modifier run3_ecal is used to change the timing algorithm to CC timing and use different records with the label 'CC' for the timing calibrations and timing offset constants.

Note that this behaviour is different than what was discussed in the meeting on the 27th Sept. when the plan was made to make the CC timing the default and use eras to modify Run 1 and Run 2 configurations to use the ratio method. Since Run 1 configurations could not be modified using eras we decided to reverse the behaviour as in the description above.

Backport of #42928

PR validation:

The PR passes Run 1 and Run2 WFs in the limited matrix tests but currently fails all Run 3 and Phase 2 WFs because the consumed records with the 'CC' label are not yet in the GTs.

[cmsbuild]

A new Pull Request was created by @thomreis (Thomas Reis) for CMSSW_13_1_X.

It involves the following packages:

• Configuration/Eras (operations)
• RecoLocalCalo/EcalRecProducers (reconstruction)

@fabiocos, @cmsbuild, @rappoccio, @mandrenguyen, @davidlange6, @jfernan2, @antoniovilela can you please review it and eventually sign? Thanks.
@fabiocos, @rchatter, @Martin-Grunewald, @missirol, @wang0jin, @makortel, @AnnikaStein, @youyingli, @apsallid, @argiro, @thomreis this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

• Backported from Use ECAL ratio timing algorithm for Run 1 and Run 2, and CC timing algorithm for Run 3 and beyond #42928

[thomreis]

type ecal

[thomreis]

backport of #42928

[smuzaffar]

@antoniovilela , bot does not understand -urgent. If you want to remove the urgent label then for now I can just delete the urgent comment make by @rappoccio

[antoniovilela]

@antoniovilela , bot does not understand -urgent. If you want to remove the urgent label then for now I can just delete the urgent comment make by @rappoccio

Yeah, I was trying out. Can I delete the comment to remove the label if needed?

[smuzaffar]

yes , you should have rights to do/update any comment :-) feel free to delete #42947 (comment) to remove the label

[antoniovilela]

yes , you should have rights to do/update any comment :-) feel free to delete #42947 (comment) to remove the label

Great, just deleted the comment.

[mandrenguyen]

I guess we need a new GT in 13_1_X as well before triggering the tests ?

[saumyaphor4252]

test parameters:

• pull_request=[13_1_X] Include ECAL Timing CC labelled tags in Run3 data and MC GTs #43017

[saumyaphor4252]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11034b/35187/summary.html
COMMIT: c4ec394
CMSSW: CMSSW_13_1_X_2023-10-13-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42947/35187/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestDQMOnlineClient-ecal_dqm_sourceclient had ERRORS

Comparison Summary

Summary:

• You potentially added 252 lines to the logs
• Reco comparison results: 19314 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3196443
• DQMHistoTests: Total failures: 7101
• DQMHistoTests: Total nulls: 6
• DQMHistoTests: Total successes: 3189314
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.023 KiB( 47 files compared)
• DQMHistoSizes: changed ( 4.53 ): -0.023 KiB JetMET/SUSYDQM
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: found differences in 4 / 46 workflows

[thomreis]

We need a backport of #42932 to pass the test.

[thomreis]

ECAL online DQM update backport to 131X to fix the unit tests: #43023

[thomreis]

test parameters:

• pull_requests=[13_1_X] Include ECAL Timing CC labelled tags in Run3 data and MC GTs #43017,Use the ratio timing method for ECAL online DQM - 131X #43023

[thomreis]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-11034b/35200/summary.html
COMMIT: c4ec394
CMSSW: CMSSW_13_1_X_2023-10-15-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42947/35200/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 494 lines to the logs
• Reco comparison results: 19308 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3196443
• DQMHistoTests: Total failures: 7095
• DQMHistoTests: Total nulls: 6
• DQMHistoTests: Total successes: 3189320
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.023 KiB( 47 files compared)
• DQMHistoSizes: changed ( 4.53 ): -0.023 KiB JetMET/SUSYDQM
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: found differences in 4 / 46 workflows

[thomreis]

As with the master PR we expect changes in the ECAL timing with this PR since before many workflows were using the default CC timing algorithm but with the ratio timing calibrations in the GTs. It is also possible that the changed timing has small effects on other distributions as well.

[mandrenguyen]

+reconstruction
Changes in line with PR description.

[rappoccio]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will be automatically merged.